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Why GAO Did This Study 

By deploying armed air marshals 
onboard selected flights, the 
Federal Air Marshal Service 
(FAMS), a component of the 
Transportation Security 
Administration (TSA), plays a key 
role in helping to protect 
approximately 29,000 domestic and 
international flights operated daily 
by U.S. air carriers. GAO was asked 
to examine (1) FAMS's operational 
approach or "concept of 
operations" for covering flights, (2) 
to what extent this operational 
approach has been independently 
evaluated, and (3) the processes 
and initiatives FAMS established to 
address workforce-related issues. 
GAO analyzed documented policies 
and procedures regarding FAMS's 
operational approach and a July 
2006 classified report based on an 
independent evaluation of that 
approach. Also, GAO analyzed 
employee working group reports 
and other documentation of 
FAMS's processes and initiatives 
for addressing workforce-related 
issues, and interviewed the FAMS 
Director, other senior officials, and 
67 air marshals (selected to reflect 
a range in levels of experience). 
This report is the public version of 
a restricted report (GAO-09-53SU) 
issued in December 2008. 



What GAO Recommends 



To improve future workforce 
satisfaction surveys, GAO 
recommends that the FAMS 
Director take actions to ensure that 
(1) survey questions and response 
options are clearly worded and 
structured and (2) the response 
rate is as high as possible. TSA 
agreed. 

To view the full product, including the scope 
and methodology, click on GAO-09-273. For 
more information, contact Steve Lord at (202) 
512-4379 or lords@gao.gov. 



What GAO Found 

Because the number of air marshals is less than the number of daily flights, 
FAMS's operational approach is to assign air marshals to selected flights it 
deems high risk — such as the nonstop, long-distance flights targeted on 
September 11, 2001. In assigning air marshals, FAMS seeks to maximize 
coverage of flights in 10 targeted high-risk categories, which are based on 
consideration of threats, vulnerabilities, and consequences. 

In July 2006, the Homeland Security Institute, a federally funded research and 
development center, independently assessed FAMS's operational approach 
and found it to be reasonable. However, the institute noted that certain types 
of flights were covered less often than others. The institute recommended that 
FAMS increase randomness or unpredictability in selecting flights and 
otherwise diversify the coverage of flights within the various risk categories. 
As of October 2008, FAMS had taken actions (or had ongoing efforts) to 
implement the Homeland Security Institute's recommendations. GAO found 
the institute's evaluation methodology to be reasonable. 

To address workforce-related issues, FAMS's previous director, who served 
until June 2008, established a number of processes and initiatives — such as 
working groups, listening sessions, and an internal Web site — for agency 
personnel to provide anonymous feedback to management on any topic. 
These efforts have produced some positive results. For example, FAMS 
revised its policy for airport check-in and aircraft boarding procedures to help 
protect the anonymity of air marshals in mission status, and FAMS adjusted 
its flight scheduling process for air marshals to support a better work-life 
balance. The air marshals GAO interviewed expressed satisfaction with FAMS 
efforts to address workforce-related issues. Further, the current FAMS 
Director, after being designated in June 2008 to head the agency, issued a 
broadcast message to all employees, expressing a commitment to continue 
applicable processes and initiatives. Also, FAMS has plans to conduct a 
workforce satisfaction survey of all employees every 2 years, building upon an 
initial survey conducted in fiscal year 2007. Although the 2007 survey 
indicated positive changes since the prior year, it was answered by 46 percent 
of the workforce, well short of the 80-percent response rate that the Office of 
Management and Budget (OMB) encourages for ensuring that results reflect 
the views of the target population. OMB guidance gives steps, such as 
extending the cut-off date for responding, that could improve the response 
rate of future surveys. Also, several of the 2007 survey questions were 
ambiguous, and response options were limited. Addressing these design 
considerations could enhance future survey results. 
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January 14, 2009 
Congressional Requesters 

With a shared responsibility for protecting the approximately 29,000 
domestic and international flights operated daily by U.S. commercial 
passenger air carriers, the Federal Air Marshal Service (FAMS) is a key 
component of the Transportation Security Administration's (TSA) layered 
approach for aviation security. 1 FAMS's overall mission is to help secure 
the nation's aviation system by detecting, deterring, and defeating hostile 
acts targeting U.S. air carriers, airports, passengers, and crews. In 
particular, FAMS's core mission is to deploy trained and armed federal air 
marshals — a workforce generally characterized as the nation's most 
mobile law enforcement officers — to provide an on-board security 
presence on selected flights. 2 

In response to the September 11, 2001, terrorist attacks and pursuant to 
the Aviation and Transportation Security Act, FAMS's mission was 
expanded to provide coverage of domestic as well as international flights, 
with priority given to nonstop, long-distance flights such as those targeted 
on September ll. 3 As a result, the agency's cadre of air marshals grew 
significantly. 4 Nonetheless, because the total number of air marshals is 
less than the approximately 29,000 domestic and international flights 
operated daily by U.S. commercial passenger air carriers, FAMS routinely 
must determine which flights are to be provided an on-board security 
presence. To facilitate making these decisions, FAMS developed an 



1 FAMS is one layer among the multiple layers of aviation security. For example, 
prospective passengers are prescreened against applicable records in the consolidated 
watch list managed by the Federal Bureau of Investigation's Terrorist Screening Center. 
Also, at TSA checkpoints in airports across the nation, passengers and baggage are 
physically screened. Air marshals generally are characterized as being the last line of 
defense within this layered aviation-security framework. 

2 Air marshals are included in the GS-1801 federal law enforcement officer job series, as 
defined by the Office of Personnel Management. Law enforcement personnel with this 
series designation engage in general inspection and compliance activities. See GAO, 
Federal Law Enforcement: Survey of Federal Civilian Law Enforcement Functions and 
Authorities, GAO-07-121 (Washington, D.C.: Dec. 19, 2006). 

3 See Pub. L. No. 107-71, § 105, 115 Stat. 597, 606-08 (2001) (codified as amended at 49 
U.S.C. § 44917). 

4 The specific number of federal air marshals is classified. 
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operational approach — commonly referred to as the agency's concept of 
operations — for deploying air marshals on selected flights. 

In addition to flight-coverage decisions, FAMS also faces challenges in 
addressing various operational and quality-of-life issues that affect the 
ability of air marshals to carry out the agency's mission — issues that have 
been noted by our prior work and also have received extensive media 
coverage. 5 Such issues range, for example, from maintaining anonymity 
during aircraft boarding procedures to mitigating the various health 
concerns associated with frequent flying. 

You requested that GAO evaluate FAMS's concept of operations and 
workforce-related issues. This report answers the following questions: 

• What is FAMS's operational approach for achieving its core mission of 
providing an onboard security presence for flights operated by U.S. 
commercial passenger air carriers? 

• To what extent has FAMS's operational approach for achieving its core 
mission been independently assessed? 

• To what extent does FAMS have processes and initiatives in place to 
address issues that affect the ability of its workforce to carry out its 
mission? 

This report is a public version of the restricted report (GAO-09-53SU) that 
we provided to you on December 8, 2008. DHS and TSA deemed some of 
the information in the restricted report as sensitive security information, 
which must be protected from public disclosure. Therefore, this report 
omits this information, such as the specific details associated with FAMS's 
concept of operations. Although the information provided in this report is 
more limited in scope, it addresses the same questions as the restricted 
report. Also, the overall methodology used for both reports is the same. 

In studying FAMS's operational approach, we reviewed relevant legislation 
regarding FAMS's mission, and we analyzed the agency's policies and 
other documentation regarding the strategy and concept of operations for 
carrying out that mission, including the procedures for assigning or 
scheduling air marshals to selected flights. Also, we reviewed the results 



6 For example, see GAO, Aviation Security: Federal Air Marshal Service Could Benefit 
from Improved Planning and Controls, GAO-06-203 (Washington, D.C.: Nov. 28, 2005). 
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of an evaluation conducted in 2003 by the Office of Management and 
Budget (OMB), which utilized its Program Assessment Rating Tool (PART) 
to assess the management and performance of FAMS. 6 Further, we also 
reviewed the follow-on reassessment of FAMS that OMB conducted in 
2008. 

To determine the extent to which FAMS's operational approach has been 
independently assessed, we reviewed a July 2006 classified report 
prepared by the Homeland Security Institute based on its independent 
evaluation of FAMS's concept of operations. 7 Our engagement team 
included a social science analyst and an economist with experience in risk 
assessment, who used generally accepted social science research 
standards in reviewing the Homeland Security Institute's report. Also, we 
interviewed applicable Homeland Security Institute officials to enhance 
our understanding of the evaluation's scope, methodology, findings, and 
recommendations. Based on our review and discussion, we determined 
the Homeland Security Institute's report to be sufficiently reliable for the 
purposes of our work. Further, we reviewed FAMS documentation — and 
interviewed the Director of FAMS and other senior officials at the agency's 
headquarters — regarding the status of efforts to address recommendations 
made by the Homeland Security Institute and any related initiatives 
involving strategic planning and the agency's concept of operations. We 
also reviewed two additional Homeland Security Institute reports, which 
FAMS provided to us in September 2008. One of the reports detailed the 
Homeland Security Institute's analysis regarding requirements for an 
improved mission scheduling tool for FAMS, 8 and the other report 



PART consists of a standard series of questions intended to determine the strengths and 
weaknesses of federal programs. The PART questions cover four broad topics — (1) 
program purpose and design, (2) strategic planning, (3) program management, and (4) 
program results/accountability. See GAO, Program Evaluation: OMB's PART Reviews 
Increased Agencies' Attention to Improving Evidence of Program Results, GAO-06-67 
(Washington, D.C.: Oct. 28, 2005). 

7 The Homeland Security Institute is a federally funded research and development center 
established pursuant to the Homeland Security Act of 2002. See Pub. L. No. 107-296, § 312, 
116 Stat. 2135, 2176, as amended. The Institute's mission is to assist the Department of 
Homeland Security address relevant issues requiring scientific, technical, and analytical 
expertise. In response to OMB's 2003 PART review of FAMS, which concluded that key 
aspects of program design needed to be independently assessed, FAMS contracted for an 
independent evaluation to be conducted by the Homeland Security Institute. 

8 Homeland Security Institute, Analysis for the Next-Generation Federal Air Marshal 
Service (FAMS) Mission Scheduling & Notification System (MSNS), final report (July 10, 
2008). 
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presented the institute's benchmark analysis that compared FAMS's 
workday rules 9 and practices against those of similar occupations 
involving frequent air travel and the related operational challenges, 
including fatigue and other human factors. 10 

Regarding processes and initiatives to address issues affecting the ability 
of air marshals to carry out the agency's mission, we reviewed prior 
reports — including our September 2005 report on planning and controls 
for FAMS. 11 We also reviewed FAMS documentation regarding various 
working groups and other initiatives that FAMS had established to address 
issues that affect the ability of air marshals to carry out the agency's 
mission. For criteria in reviewing the agency's documentation regarding 
these efforts, we drew on our prior work regarding leading organizations 
and the best practices for strategically managing human capital. 12 Further, 
we interviewed the Director of FAMS and other senior officials at agency 
headquarters, and we visited 11 of the agency's 21 field offices, where we 
interviewed managers and a total of 67 air marshals. We selected the 11 
field offices and the 67 air marshals based on nonprobability sampling. 13 
Regarding field office selections, we considered various factors, such as 
geographic location and the involvement of local management in 
agencywide working groups to address issues affecting air marshals. Our 
selections of air marshals were made to encompass a variety of experience 
levels. Because we selected a nonprobability sample of FAMS field offices 
to visit and air marshals to interview, the information we obtained in these 
visits and interviews cannot be generalized either to all 21 field locations 
or to all air marshals in the offices we visited. However, the interviews 



9 "Workday rules" refer to the parameters that FAMS uses for assigning air marshals to 
flights. 

10 Homeland Security Institute, Federal Air Marshal Service (FAMS) Risk-Based Resource 
Allocation Study: Human Factors Analysis, final report (July 10, 2008). 

11 GAO, Aviation Security: Federal Air Marshal Service Could Benefit from Improved 
Planning and Controls, GAO-05-884SU (Washington, D.C.: Sept. 29, 2005). The report is 
restricted (not available to the public) because it contains sensitive security information. 
The public version of the report is GAO-06-203 (Nov. 28, 2005). 

12 See GAO, A Model of Strategic Human Capital Management, GAO-02-373SP 
(Washington, D.C.: Mar. 15, 2002). 

13 Nonprobability sampling is a method of sampling where observations are selected in a 
manner that is not completely random, generally using specific characteristics of the 
population as criteria. Results from a nonprobability sample cannot be used to make 
inferences about an entire population because some elements of the population being 
studied had no chance or an unknown chance of being selected as part of the sample. 
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provided us a broad overview of issues important to air marshals. We also 
reviewed documentation regarding the implementation and results of a 
workforce satisfaction survey that FAMS conducted in 2007. Our 
engagement team, which included social science analysts with extensive 
survey research experience, reviewed the questionnaire used in the survey 
for clarity and the related response options for appropriateness. Also, we 
discussed with FAMS officials the extent to which efforts were made to 
obtain an overall response rate as high as possible. As criteria to guide our 
review of the survey results, we used OMB's Standards and Guidelines for 
Statistical Surveys (September 2006) and related guidance applicable to 
federal surveys. 

We conducted this performance audit from April 2007 to December 2008 in 
accordance with generally accepted government auditing standards. Those 
standards require that we plan and perform the audit to obtain sufficient, 
appropriate evidence to provide a reasonable basis for our findings and 
conclusions based on our audit objectives. We believe that the evidence 
obtained provides a reasonable basis for our findings and conclusions 
based on our audit objectives. More details about the scope and 
methodology of our work are presented in appendix I. 



RGSllltS in Brief FAMS's operational approach (or concept of operations) is based on 

" * ^ assessments of threat, vulnerability, and consequences to help ensure that 

high-risk flights operated by U.S. commercial air carriers — such as the 
nonstop, long-distance flights targeted on September 11, 2001 — are given 
priority coverage by federal air marshals. Since it is not feasible for federal 
air marshals to cover all of the approximately 29,000 domestic and 
international flights operated daily by U.S. commercial passenger air 
carriers, FAMS aims to assign air marshals to selected, higher-risk flights. 
Under this approach, FAMS seeks to maximize coverage of high-risk 
flights in 10 targeted critical flight categories, such as flights to and from 
the national capital region and long-distance flights from various other 
locations. FAMS relies on a scheduling process for determining the most 
efficient flight combinations that will allow air marshals to cover the 
desired flights. According to FAMS officials, the overall coverage goals 
and the corresponding flight schedules of air marshals are subject to 
modification at any time based on changing threat information and 
intelligence. 

In July 2006, the Homeland Security Institute, a federally funded research 
and development center, concluded that FAMS's operational approach for 
providing an on-board security presence on high-risk flights was 
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reasonable. However, the Homeland Security Institute noted that certain 
types of flights were covered less often than others. The Homeland 
Security Institute recommended that FAMS increase randomness or 
unpredictability in selecting flights and to otherwise diversify the coverage 
of flights. As of October 2008, FAMS had implemented or had ongoing 
efforts to implement the institute's recommendations. For example, FAMS 
is developing an automated decision-support tool for selecting flights; this 
effort is expected to be completed by December 2009. We reviewed the 
Homeland Security Institute's evaluation methodology and found it to be 
reasonable. 

To better understand and address operational and quality-of-life issues 
affecting the FAMS workforce, the agency's previous director — who 
served in that capacity from March 2006 to June 2008 — established various 
processes and initiatives, including 36 issue-specific working groups 
composed of managers and air marshals. The previous director also 
established listening sessions that provided a forum for employees to 
communicate directly with senior management and an internal Web site 
for agency personnel to provide anonymous feedback to management. 
These efforts have produced some positive results. For example, FAMS 
amended its policy for airport check-in and flight boarding procedures 
(effective May 15, 2008) to better ensure the anonymity of air marshals in 
mission status 14 and also adjusted its flight scheduling process for air 
marshals to support a better work-life balance. All of the 67 air marshals 
we interviewed in 11 field offices commented favorably about these efforts 
and credited the leadership of the previous FAMS Director. To reinforce a 
shared vision for workforce improvements and to sustain forward 
progress, the current FAMS Director has expressed a commitment to 
continuing applicable processes and initiatives. 

Also, FAMS has plans to conduct a workforce satisfaction survey of all 
employees every 2 years, building upon an initial survey conducted in 
fiscal year 2007. A majority (79 percent) of the respondents to the 2007 
survey indicated that there had been positive changes from the prior year, 
although the overall response rate (46 percent) constituted less than half 
of the FAMS workforce and was substantially less than the 80 percent 
encouraged by OMB in its guidance for federal surveys that require its 



FAMS's changes to check in and boarding procedures concern air marshals' interactions 
with airline personnel. FAMS's policy continues to require air marshals to adhere to 
established TSA regulations and locally established airport procedures. 
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approval. According to the OMB guidance, a high response rate increases 
the likelihood that the views of the target population are reflected in the 
survey results. Although the FAMS workforce survey did not require OMB 
approval, we believe that OMB guidance provides direction in planning, 
designing, and implementing surveys. While FAMS made some efforts to 
obtain responses to the 2007 survey, other methods outlined in the OMB 
guidance, such as extending the cut-off date for responding, could improve 
the response rate of future FAMS surveys. 15 Also, 7 of the 60 questions in 
the 2007 survey questionnaire combined two or more issues, which could 
cause respondents to be unclear on what issue to address and result in 
potentially misleading responses. Moreover, none of the 60 questions in 
the 2007 survey questionnaire provided for response options such as "not 
applicable" or "no basis to judge" — responses that would be appropriate 
when respondents had little or no familiarity with the topic in question. 
Thus, the 2007 survey's results may not provide a complete assessment of 
employees' satisfaction. 

To increase the usefulness of biennial workforce satisfaction surveys, we 
are recommending that the FAMS Director take steps to ensure that the 
surveys are well designed and that additional efforts are considered for 
obtaining the highest possible response rates. In November 2008, in 
written comments on a draft of our restricted report, the Department of 
Homeland Security and TSA agreed with our recommendation and noted 
that FAMS was in the initial stages of formulating the next workforce 
satisfaction survey, which included plans to implement the 
recommendation. 



BcLCkSrOlind Originally established as the Sky Marshal program in the 1970s to counter 

" hijackers, the Aviation and Transportation Security Act expanded FAMS's 

mission and workforce in response to the September 11, 2001, terrorist 
attacks and mandated the deployment of federal air marshals on high- 
security risk flights. Within the 10-month period immediately following 
September 11, 2001, the number of air marshals grew significantly. Also, 
during years following the 2001 attacks, FAMS underwent various 



15 FAMS officials attributed the 2007 survey's 46-percent response rate to the highly mobile 
nature of the agency's workforce and also noted that this rate is consistent with the 
response rates for similar workforce satisfaction surveys in the federal government. FAMS 
officials added that they view the workforce survey as one of a variety of tools — along with 
the numerous issue-specific working groups and other processes and initiatives — for 
raising issues of concern and providing feedback on actions taken by management. 
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organizational transfers. Initially, FAMS was transferred within the 
Department of Transportation from the Federal Aviation Administration to 
the newly created TSA. In March 2003, FAMS moved, along with TSA, to 
the newly established Department of Homeland Security. In November 
2003, FAMS was transferred to U.S. Immigration and Customs 
Enforcement (ICE). Then, about 2 years later, FAMS was transferred back 
to TSA in the fall of 2005. 16 

FAMS is one layer among multiple layers of aviation security. For 
example, prospective passengers are prescreened against applicable 
records in the Terrorist Screening Center's consolidated watch list. 17 
Passengers and baggage are also physically screened. Air marshals 
generally are characterized as being the last line of defense within the 
layered aviation-security framework. In this regard, FAMS officials 
stressed that air marshals constitute the only in-flight security layer 
deployed on the basis of risk. 18 

FAMS deploys thousands of federal air marshals to a significant number of 
daily domestic and international flights. In carrying out this core mission 
of FAMS, air marshals are deployed in teams to various passenger flights. 19 



The transfer of FAMS to ICE was premised partly on the assumptions that (1) air 
marshals would be afforded a broader career path by cross-training with ICE's investigative 
division and (2) ICE's special agents could provide a surge capability by serving as 
supplemental air marshals, if needed. See GAO, Aviation Security: Federal Air Marshal 
Service Is Addressing Challenges of Its Expanded Mission and Workforce, but Additional 
Actions Needed, GAO-04-242 (Washington, D.C.: Nov. 19, 2003). 

17 The Terrorist Screening Center, managed by the Federal Bureau of Investigation, 
maintains a consolidated watch list of known or appropriately suspected terrorists and 
sends records from the list to agencies to support terrorism-related screening. TSA 
currently provides airlines with portions of the Terrorist Screening Center's watch list — the 
No Fly and Selectee lists — and directs them to screen the names of passengers to identify 
those who may pose threats to aviation. In general, individuals on the No Fly List are to be 
precluded from boarding an aircraft, and individuals on the Selectee List are to receive 
additional physical screening prior to boarding an aircraft. See GAO, Terrorist Watch List 
Screening: Opportunities Exist to Enhance Management Oversight, Reduce 
Vulnerabilities in Agency Screening Processes, and Expand Use of the List, GAO-08-110 
(Washington, D.C.: Oct. 11, 2007). 

18 Other in-flight security layers not employed based on risk include hardened cockpit 
doors and armed flight crew members who are part of TSA's Federal Flight Deck Officer 
program. 

19 The specific number of air marshals assigned to an on-board team, whether for a 
domestic flight or an international flight, may vary depending on such factors as duration of 
the flight, the type of aircraft, the departure and destination cities, and awareness of 
specific threat information. 
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Such deployments are based on FAMS's concept of operations, which 
guides the agency in its selection of flights to cover. Once flights are 
selected for coverage, FAMS officials stated that they must schedule air 
marshals based on their availability, 20 the logistics of getting individual air 
marshals in position to make a flight, and applicable workday rules. 

At times, air marshals may have ground-based assignments. On a short- 
term basis, for example, air marshals participate in Visible Intermodal 
Prevention and Response (VIPR) teams, which provide security 
nationwide for mass transit systems other than aviation. 21 Also, air 
marshals participate in Joint Terrorism Task Forces led by the Federal 
Bureau of Investigation. 

Good marksmanship is considered a necessity for air marshals, 
particularly given the unique environment of the core mission — the 
relatively tight confines of an airplane, coupled with the presence of 
numerous passengers ("bystanders") and the possibility of air turbulence 
that creates an unstable "shooting platform" (see fig. 1). Thus, according to 
TSA, air marshals have the highest marksmanship standard in the federal 
government and also must be recertified on their firearm every quarter. 



In determining air marshals' availability, FAMS officials stated that they must consider 
such factors as training requirements, other ground-based duties, and annual leave plans. 

21 After the March 2004 train bombings in Madrid, TSA created and deployed Visible 
Intermodal Prevention and Response teams to enhance security on U.S. rail and mass 
transit systems nationwide. Comprised of federal air marshals, surface transportation 
security inspectors, transportation security officers, behavior detection officers, and 
explosives detection canines, the teams are intended to work with local security and law 
enforcement officials to supplement existing security resources, provide a deterrent 
presence and detection capabilities, and introduce an element of unpredictability to disrupt 
potential terrorist activities. 
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Figure 1 : Federal Air Marshals Are Trained for a Unique Working Environment 




Source: GAO and the Federal Air Marshal Service (FAMS). 

Note: Faces are intentionally blurred. 



To preserve their anonymity on covered flights, air marshals are to blend 
in with other passengers by dressing appropriately and performing their 
duties discreetly without drawing undue attention. 



The Federal Air 
Marshal Service's 
Operational Approach 
to Achieving Its Core 
Mission Is Based on 
Risk-Related Factors 



FAMS's operational approach (concept of operations) is based on risk- 
related factors, such as assessments of threat, vulnerability, and 
consequences. FAMS is guided by the provisions of the Aviation and 
Transportation Security Act that specify the deployment of federal air 
marshals on flights presenting high-security risks, such as nonstop, long- 
distance flights targeted on September 11, 2001. FAMS seeks to maximize 
coverage of high-risk flights by establishing coverage goals for 10 targeted 
critical flight categories. In order to reach these coverage goals, FAMS 
uses a scheduling process to determine the most efficient flight 
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combinations that will allow air marshals to cover the desired flights. 
FAMS management officials stressed, however, that the overall coverage 
goals and the corresponding flight schedules of air marshals are subject to 
modification at any time based on changing threat information and 
intelligence. 



Risk-Related Factors Are Following the attacks of September 11, 2001, FAMS developed a risk- 
Considered in Assigning based concept of operations for deploying air marshals on U.S. 
Air Marshals to Flights commercial passenger air carriers. Because there are many more U.S. air 

55 carrier flights each day than can be covered by air marshals, FAMS relies 

on the methodology outlined in its concept of operations to assign air 
marshals to flights with the highest security risks. Under this approach, 
FAMS considers the following risk-related factors to categorize each of the 
approximately 29,000 domestic and international flights operated daily by 
U.S. commercial passenger air carriers into risk categories — high risk or 
lower risk: 22 



• Threat (intelligence): Available strategic or tactical information 
affecting aviation security is considered. 23 



• Vulnerabilities: Although FAMS's specific definition is deemed to be 
sensitive security information, DHS defines "vulnerability" as a 
physical feature or operational attribute that renders an entity open to 
exploitation or susceptible to a given hazard. 



• Consequences: FAMS recognizes that flight routes over certain 

geographic locations involve more potential consequences than other 
routes. 

FAMS attempts to assign air marshals to provide an on-board security 
presence on as many of the flights in the high-risk category as possible. 
However, other considerations can make covering only high-risk flights 
impractical from a scheduling perspective and potentially predictable to 
an adversary. Therefore, for purposes of scheduling efficiency and 
adversary uncertainty, FAMS may deploy some air marshals on lower-risk 
flights. 



FAMS's criteria for determining high-risk flights are classified. 

FAMS considers "threat" and "intelligence" as separate risk-related factors. 
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Air Marshals Are Deployed 
on Flights to Meet 
Coverage Goals for 
Targeted Critical Flight 
Categories; the Concept of 
Operations Provides 
Flexibility for Responding 
to Changing Threats 



FAMS has established a scheduling process intended to maximize the 
coverage of high-risk flights and meet the agency's desired coverage goals 
for 10 targeted critical flight categories. 24 FAMS's Domestic Planning 
Branch (within the Systems Operation Control Division) is responsible for 
scheduling air marshals to domestic missions. During the course of a year, 
the Domestic Planning Branch must prepare schedules for 13 roster 
periods of 28 days each. According to FAMS officials, each 28-day 
schedule takes approximately 3 weeks to prepare. The Domestic Planning 
Branch prepares each domestic schedule using an automated scheduling 
tool. 25 As part of the scheduling process, each FAMS field office is 
responsible for making available a specific percentage of their air marshals 
on a daily basis to cover targeted critical flights (both domestic and 
international flights) in the roster periods. FAMS utilizes the automated 
scheduling tool to determine the most efficient flight "pairings" of 
departure and return flights that will bring an air marshal back to his or 
her starting point and that would be within the parameters for mission 
assignment and rest for the air marshal. FAMS officials also perform other 
checks on the fairness or appropriateness of the schedules, such as 
ensuring that certain flights are not being covered repeatedly by the same 
air marshals. 

FAMS officials noted that the schedules for deploying air marshals are 
altered as needed to cover specific, high-threat flights. For example, in 
August 2006, FAMS increased its coverage of international flights in 
response to the discovery, by authorities in the United Kingdom, of 
specific terrorist threats directed at flights from Europe to the United 
States. However, the officials added that a shift in resources of this type 
can have consequences because of the limited number of air marshals. 
FAMS officials noted that international missions require more resources 
than domestic missions partly because the trips are of longer duration. 



FAMS's coverage rate goals for the 10 targeted critical flight categories are classified. 

25 According to FAMS officials, international mission schedules are prepared by Systems 
Operation Control Division officials (other than those in the Domestic Planning Branch) 
who use manual methods because of added complexities, such as the need to coordinate 
with foreign governments and make reservations at overseas locations. 
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In its 2003 PART review of FAMS, OMB concluded that an independent 
evaluation should be conducted to assess FAMS's performance related to 
aspects of the agency's concept of operations — particularly aspects 
involving flight coverage risk categories, the distribution of covered 
flights, and target levels of coverage. 26 The Homeland Security Institute, a 
federally funded research and development center, performed this 
evaluation and issued a final report in July 2006. The report concluded that 
FAMS's approach for achieving its core mission of providing an onboard 
security presence for flights, as detailed in the agency's concept of 
operations, was reasonable and made several recommendations for 
enhancements. 27 The Homeland Security Institute recommended, for 
example, that FAMS increase randomness or unpredictability in selecting 
flights and otherwise diversify the coverage of flights within various risk 
categories. As of October 2008, FAMS had implemented or had ongoing 
efforts to implement the recommended enhancements. 

In a July 2006 report, the Homeland Security Institute specifically noted 
the following regarding FAMS's overall approach to flight coverage: 

• FAMS applies a structured, rigorous approach to analyzing risk and 
allocating resources. 

• The approach is reasonable and valid. 

• No other organizations facing comparable risk-management challenges 
apply notably better methodologies or tools. 

As part of its evaluation methodology, the Homeland Security Institute 
examined the conceptual basis for FAMS's approach to risk analysis. Also, 
the institute examined FAMS's scheduling processes and analyzed outputs 
in the form of "coverage" data reflecting when and where air marshals 
were deployed on flights. Further, the Homeland Security Institute 
developed and used a model to study the implications of alternative 
strategies for assigning resources. We reviewed the Homeland Security 
Institute's evaluation methodology and generally found it to be reasonable. 

In a 2008 PART reassessment of FAMS, OMB also reported that the 
Homeland Security Institute's evaluation employed quality evaluation 
methods and was comprehensive in scope. Further, OMB noted that an 



An Independent 
Assessment 
Concluded That the 
Federal Air Marshal 
Service's Approach 
for Achieving Its Core 
Mission Was 
Reasonable; 
Recommendations for 
Enhancing the 
Approach Are Being 
Implemented 



26 Appendix II presents more details about the results of OMB's 2003 PART review of FAMS 
and the follow-on reassessment conducted in 2008. 

27 Much of the specific information in the report is classified. 
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interagency steering group — which was convened by the Homeland 
Security Institute and met in conference in April 2006 — also had reviewed 
FAMS's concept of operations and considered it to be reasonable. In 
addition to FAMS and Homeland Security Institute participants, the 
interagency steering group consisted of representatives from various law 
enforcement and counterterrorism agencies, which included the Federal 
Bureau of Investigation, U.S. Customs and Border Protection, the 
Transportation Security Administration, the Federal Aviation 
Administration, the Homeland Infrastructure Threat and Risk Assessment 
Center, 28 U.S. Northern Command/North American Aerospace Defense 
command, 29 and the National Counterterrorism Center. 

In its July 2006 report, the Homeland Security Institute made several 
recommendations for enhancing FAMS's approach for deploying air 
marshals on flights. As presented in table 1, FAMS had implemented or 
had ongoing efforts to implement all of the recommended enhancements, 
as of October 2008. 



Table 1: Homeland Security Institute's Recommendations (July 2006 Report) to Improve the Federal Air Marshal Service's 
Approach for Achieving Its Core Mission 



Topic Discussion, specific recommendations, and implementation status (as of October 2008) 

Definition of vulnerability Discussion: According to the Homeland Security Institute, FAMS's definition of "vulnerability" is 

inconsistent with traditional risk-based methodologies. 

Recommendation: FAMS should engage the aviation security community to better understand 
vulnerabilities surrounding aircraft configuration or possible gaps in the air defense system. 

Implementation status: FAMS officials stated that they consulted with the aviation security community 
and believe that the agency's definition of vulnerability is appropriate for risk analyses relevant to the 
mission of FAMS. However, the officials also noted that a new automated decision-support tool for 
selecting flights is being developed with assistance from the Homeland Security Institute and will 
incorporate consideration of more traditional aspects of vulnerability. FAMS expects development of 
this tool to be completed by the end of calendar year 2009. 



The center was established by the Department of Homeland Security. The center's 
intelligence analysts and infrastructure specialists work to identify the threat to critical 
infrastructures, vulnerabilities and interdependencies, and the overall risk inherent in any 
potential attack against critical infrastructure. 

29 The U.S. Northern Command provides command and control of Department of Defense 
homeland defense efforts and coordinates defense support of civil authorities. The North 
American Aerospace Defense Command is the binational American and Canadian 
command responsible for the air defense of North America and maritime warning for the 
United States and Canada. 



Page 14 



GAO-09-273 Federal Air Marshal Service 



Topic 



Discussion, specific recommendations, and implementation status (as of October 2008) 



Filtering process Discussion: The Homeland Security Institute reported that FAMS's interpretation and implementation of 

impact (consequences) and the subsequent "filtering" process heavily bias the agency's allocation 
decisions. 

Recommendation: FAMS should engage the intelligence and security communities to validate these 
basic assumptions concerning threat and consequence. 

Implementation status: FAMS has modified its flight filtering approach to open up more flights for 
potential coverage. In addition, the Homeland Security Institute, as part of its ongoing work with FAMS 
after issuance of the July 2006 report, continued to examine FAMS's approach to flight coverage and 
will provide FAMS a report by the end of calendar year 2009. 



Allocation process Discussion: The Homeland Security Institute found that the allocation algorithm used by FAMS is 

biased and may not be consistent with effective risk reduction. 

Recommendation: FAMS should continue pursuing next-generation scheduling and analysis tools to 
better quantify scheduling efficiency and risk-reduction effectiveness. 

Recommendation: To compensate for a lack of resources and to deny predictability, FAMS should 
integrate randomness (unpredictability) in its scheduling process. 

Implementation status: As another part of its ongoing work, the Homeland Security Institute is assisting 
FAMS in modifying the current scheduling tool to provide more randomness in flight scheduling. In July 
2008, the institute provided FAMS a report detailing requirements for a new scheduling tool. The 
institute's report recommended that FAMS continue to use its present automated scheduling tool but 
expand the capabilities of the software for determining flight risk and allocating resources based on 
that risk. In addition, the institute recommended an approach for using the scheduling tool to increase 
the randomness of flights selected for coverage — that is, an approach designed to randomly select 
some flights in each risk category while selecting other flights in the respective category based on 
efficient flight pairings and the need for scheduling efficiency. The institute also recommended that 
FAMS continue to research methods for incorporating more randomness into its scheduling process. 
FAMS officials indicated that these ongoing efforts with the institute, along with the modifications to the 
filtering process mentioned above, will help address the randomness issue. FAMS officials stated that 
a date for final completion of the new scheduling tool is uncertain since it is dependent on the agency's 
future budgets. 



Performance measures Discussion: The Homeland Security Institute reported that FAMS's primary performance measure — 

average coverage rates — can mask weaknesses in coverage patterns (variance). 3 

Recommendation: FAMS should develop performance measures to track the distribution or coverage 
across certain categories of flights. 

Implementation status: FAMS contracted with the Homeland Security Institute to develop, as part of its 
ongoing work, proposals for alternative performance measures for FAMS to consider. The institute's 
final report is due to FAMS by the end of calendar year 2008. 
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Topic 



Discussion, specific recommendations, and implementation status (as of October 2008) 



Decision-support tools Discussion: During the course of its analysis, the Homeland Security Institute noted that FAMS's 

decision makers did not have a tool for evaluating the resource implications of different threat 
scenarios and alternative coverage schemes. 

Recommendation: FAMS should consider building a decision-support tool, along the lines of the model 
that the Homeland Security Institute developed based on risk balancing, to facilitate a system-wide 
view of resource decisions. 

Implementation status: FAMS, with the assistance of the Homeland Security Institute, is developing an 
automated decision-support tool that will be linked to its flight-scheduling system. FAMS expects the 
new decision-support tool to consider every flight over critical infrastructure and include input for 
vulnerability of different types of aircraft and airports. Development of the decision-support tool is 
anticipated to be completed by the end of calendar year 2009. In the interim, FAMS is basing its 
resource-allocation decisions partly on manual processes whereby FAMS officials review the threat 
environment and determine the most beneficial mix of international and domestic flights, while 
observing budgetary and personnel constraints. 

Source: Homeland Security Institute; GAO analysis. 

Note: The "discussion" and the "recommendation" narratives are excerpts from the Homeland 
Security Institute's July 2006 report. For more details about the institute's report, see appendix III. 

"Appendix IV presents more details about FAMS's key performance measure. 

In reference to the core mission of FAMS, the Homeland Security 
Institute's recommendations regarding two processes — the filtering 
process for selecting flights and the allocation process for assigning air 
marshals to flights — are particularly important. To address the institute's 
recommendations, FAMS officials stated that a broader approach to 
filtering flights has been implemented — an approach that opens up more 
flights for potential coverage, provides more diversity and randomness in 
flight coverage, and extends flight coverage to a variety of airports. 

The Homeland Security Institute's ongoing work has also resulted in two 
reports delivered to FAMS in July 2008. One of the reports detailed the 
institute's analysis regarding requirements for a next-generation-mission 
scheduling tool for FAMS, 30 and the other report presented the institute's 
benchmark analysis that compared FAMS's workday rules and practices 
against those of similar occupations involving frequent air travel and the 
related operational challenges, including fatigue and other human 
factors. 31 Also, in September 2008, the Homeland Security Institute 
provided FAMS a classified report assessing the deterrent effects of the 



Homeland Security Institute, Analysis for the Next-Generation Federal Air Marshal 
Service (FAMS) Mission Scheduling & Notification System (MSNS), final report (July 10, 
2008). 

31 Homeland Security Institute, Federal Air Marshal Service (FAMS) Risk-Based Resource 
Allocation Study: Human Factors Analysis, final report (July 10, 2008). 
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agency's approach to flight coverage. Further, based on its continuing 
work, the institute expects to provide FAMS one additional final report by 
the end of calendar year 2008 — a report regarding potential enhancements 
to performance measures. 32 



The Federal Air 
Marshal Service Has 
Taken Positive 
Actions to Address 
Issues Affecting Its 
Workforce and to 
Help Ensure 
Continued Progress 



To identify and address issues affecting the ability of its workforce to 
successfully carry out its mission, FAMS has implemented various 
communication-oriented processes or initiatives — including 36 issue- 
specific working groups — that have produced some positive results. For 
instance, FAMS has revised and documented certain policies — including 
the policy related to aircraft check-in and boarding procedures — to better 
protect air marshals' anonymity. In addition, FAMS has modified its 
mission scheduling processes and implemented a voluntary lateral transfer 
program to address certain issues regarding air marshals' quality of life — 
and has plans to further address health issues associated with varying 
work schedules and frequent flying. As an additional initiative to help 
determine the effectiveness of management's actions to address issues 
affecting air marshals, FAMS conducted a workforce satisfaction survey of 
all staff in late fiscal year 2007. A majority (79 percent) of the respondents 
to the survey indicated that there had been positive changes from the prior 
year, although the overall response rate (46 percent) constituted less than 
half of the workforce. The 46 percent response rate was substantially less 
than the 80 percent rate encouraged by OMB in its guidance for federal 
surveys that require its approval. According to the OMB guidance, a high 
response rate increases the likelihood that the views of the target 
population are reflected in the survey results. Obtaining a higher response 
rate to FAMS's future surveys, which the agency plans to conduct every 2 
years, and modifying the structure of some questions, could enhance the 
surveys' potential usefulness by, for instance, providing a more 
comprehensive basis for assessing employees' attitudes and perspectives. 
All 67 of the air marshals we interviewed in 11 field offices attributed 
progress under these efforts largely to the "tone at the top," particularly 
the commitment exhibited by the former FAMS Director who served in his 
position from March 2006 to June 2008. To reinforce a shared vision for 
workforce improvements and sustain forward progress, the current FAMS 
Director has expressed a commitment to continuing applicable processes 
and initiatives. 



' The Institute noted that it is coordinating its work on performance measures with OMB. 
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The Federal Air Marshal 
Service Implemented 
Various Processes or 
Initiatives to Address 
Issues Affecting Its 
Workforce 



Our prior work has shown that leading organizations commonly sought 
their employees' input on a periodic basis — by, for example, establishing 
working groups or task forces, convening focus groups, and conducting 
employee satisfaction surveys — and used that input to adjust their human 
capital approaches. 33 Starting in March 2006, the then-serving FAMS 
Director implemented several communication processes or initiatives to 
better understand and address issues facing the agency's workforce. Chief 
among these processes or initiatives were issue-specific working groups 
established to study, analyze, and address a variety of issues ranging from 
mission, organizational, and operational topics to workforce satisfaction 
and quality-of-life concerns. Initially, based on his knowledge of issues 
facing the organization when he assumed the leadership position in March 
2006, the FAMS Director established 12 working groups. Subsequently, 
based on feedback from these initial groups and other sources regarding 
issues of concern, the number of working groups expanded to 36 (see app. 
V). 



Each working group typically included a special agent-in-charge, a subject 
matter expert, air marshals, and mission support personnel from the field 
and headquarters. FAMS management directed working group members to 
define each group's purpose, analyze specific issues, develop short- and 
long-term recommendations and determine their financial feasibility. As a 
final product, FAMS management expected each working group to submit 
a report, including recommendations, to the FAMS executive staff for 
managerial consideration. 34 According to FAMS management, the working 
groups typically disband after submitting a final report. FAMS 
management stressed, however, that applicable groups could be 
reconvened or new groups established as needed to address relevant 
issues. 



In addition to the working groups, other processes or initiatives 
implemented by FAMS management to address workforce issues or 



GAO, A Model of Strategic Human Capital Management, GAO-02-373SP (Washington, 
D.C.: Mar. 15, 2002). 

34 Regarding the status of the 36 working groups as of October 1, 2008, FAMS officials stated 
that (a) 18 working groups had completed their work and submitted final reports, which 
have been distributed by the FAMS Director to the workforce; (b) 9 working groups had 
submitted final reports that were still under review by FAMS management; (c) 5 working 
groups had yet to complete their work and prepare final reports; and (d) 4 working groups 
are to remain ongoing and present findings when applicable and by appropriate means. 
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otherwise improve management-workforce communication include the 
following: 

• Field office focus groups — Each of the 21 FAMS field offices organized 
a local focus group composed of representatives from the respective 
office's air marshal squads and at least one mission support staff. All 
members serve on a rotating basis, and the groups are to meet at least 
quarterly to discuss issues of concern to the local workforce and bring 
these issues to the attention of the applicable field office's special 
agent-in-charge. 

• Field office visits by the FAMS Director— In 2006, the FAMS Director 
began visiting field offices and holding informal gatherings with air 
marshals, outside the presence of local managers, to discuss their 
questions and concerns. 

• Listening sessions — FAMS senior management established forums to 
allow direct communication between FAMS senior management and 
various personnel. In 2006, the FAMS Director and Deputy Directors 
conducted these sessions weekly in headquarters and the field offices 
with a total of 10 tol4 staff selected for each meeting. In 2007, this 
format changed from weekly to monthly sessions and included larger 
groups of FAMS personnel. 

• Dinners with the Director — In 2006, the FAMS Director began holding 
weekly dinners to meet with air marshals transiting through the 
Washington, D.C., area. These dinners provide an opportunity for air 
marshals to speak personally with the director about any questions or 
concerns. The FAMS Deputy Director and one assistant director also 
attend these dinners with selected air marshals. 

• Director's e-mail in-box 35 — FAMS established an e-mail in-box for 
agency personnel to provide feedback to the FAMS Director. At any 
time, air marshals — whether at headquarters, in a field office, or 
deployed on mission — can send their insights, ideas, suggestions, and 
solutions to the FAMS Director. 

• Anonymous Web site — FAMS established an internal Web site for 
agency personnel to provide anonymous feedback to FAMS 
management on any topic. 



° FAMS also refers to the e-mail in-box as the "working group e-mail address." 
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• Ombudsman Position — FAMS management assigned an air marshal to 
the position of Ombudsman in October 2006. According to FAMS 
management, the Ombudsman provides confidential, informal, and 
neutral assistance to employees to address workplace-related 
problems, issues, and concerns. FAMS reported that, in fiscal year 
2007 (the first full year of the position), the Ombudsman handled 67 
cases, and, through the first three quarters of fiscal year 2008, an 
additional 54 cases. 

FAMS officials estimated that, as of October 2008, more than one-fourth of 
the agency's employees had participated in one or more of these activities 
which encompass the various working groups and other processes and 
initiatives. 36 

Based on input provided by the working groups and information obtained 
through the other processes and initiatives, FAMS has taken or is planning 
to take actions to address issues that affect the ability of air marshals to 
carry out the agency's mission. As discussed in the following sections, 
these actions address operational issues, such as check-in and boarding 
procedures that affect air marshals' anonymity as well as quality-of-life and 
health issues. 



To Help Preserve the 
Anonymity of Air 
Marshals, the Federal Air 
Marshal Service Revised 
Its Policy and Procedures 
for Check-In and Boarding 
of Aircraft 



To preserve their anonymity on covered flights, federal air marshals are to 
blend in with other passengers by dressing appropriately and performing 
their duties discreetly without drawing undue attention. In past years, air 
marshals frequently asserted that the check-in and boarding policy and 
procedures established by FAMS compromised their anonymity by 
requiring repeated interactions with airline personnel. In September 2005, 
we reported that the full extent of incidents that air marshals encounter 
was unknown because FAMS lacked adequate management controls for 
ensuring that such incidents were recorded, tracked, and addressed. 37 



In addition to the working groups and other processes and initiatives, in late fiscal year 
2007, FAMS conducted the first in a series of biennial surveys of its workforce. FAMS 
expected the 2007 survey to help identify issues affecting the ability of air marshals and 
other agency personnel to perform their jobs and also to help management assess the 
effectiveness of actions taken to address various issues. We discuss the 2007 survey in 
greater detail later in this report. 

37 GAO-05-884SU (Sept. 29, 2005), which is a restricted report (not available to the public) 
because it contains sensitive security information. The public version of the report is 
GAO-06-203 (Nov. 28, 2005). 
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Accordingly, to facilitate management of incidents that affect air marshals' 
ability to operate discreetly during their missions, our September 2005 
report recommended that FAMS take the following four actions: 

• Develop a means for recording all incidents reported to the Mission 
Operations Center that affect air marshals' ability to operate discreetly 
and criteria for determining which incidents require federal air 
marshals to complete a mission report. 

• Develop a means for tracking and retrieving data on mission reports to 
enable FAMS to analyze and monitor reported and systemic incidents. 

• Establish written policies and procedures for reviewing and addressing 
reported incidents. 

• Establish a means for providing feedback on the status and outcome of 
FAMS mission reports to the federal air marshals who submit them. 

FAMS has taken steps to address all four of these recommendations and 
also address the related feedback received from air marshals through 
various working groups and other initiatives. In October 2005, FAMS 
issued a written directive establishing policies and procedures for 
reporting and managing mission incidents. 38 In November 2005, we 
reported that we had reviewed the directive and believed that it addressed 
two of our recommendations — the first and the third recommendations. 39 
More recently, in March 2008, FAMS issued an addendum to its written 
directive establishing a means for providing feedback on the status and 
outcome of FAMS mission reports to the federal air marshals who submit 
them (fourth recommendation). 

Additionally, FAMS revised its policy and procedures regarding interaction 
with airline personnel during the check-in and boarding processes in order 
to better protect the anonymity of air marshals in mission status. 40 To help 
ensure effective implementation, the new policy and procedures have 
been incorporated into TSA's Aircraft Operator Standard Security 
Program, which specifies requirements that domestic passenger air 
carriers must implement as part of their TSA-approved security programs. 
According to FAMS officials, the recent update constitutes the first time 



FLD 7100, Subject: FAMS Reporting System Policy and Procedure (Oct. 18, 2005). 

39 GAO-06-203 (Nov. 28, 2005). 

40 FAMS's changes to check in and boarding procedures concern air marshals' interactions 
with airline personnel. FAMS's policy continues to require that air marshals adhere to 
established TSA regulations and locally established airport procedures. 
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that the Aircraft Operator Standard Security Program guidance specifically 
includes a section regarding the boarding of federal air marshals. 

Through use of a database created in fiscal year 2006 to track mission 
incidents, FAMS senior executive staff noted that analysis and monitoring 
are conducted daily of reported incidents, including those that could 
compromise the ability of air marshals to operate discreetly (second 
recommendation). The first management report detailing overall incident 
patterns and trends was produced in July 2008. Going forward, FAMS 
officials stated that reports would be produced quarterly to allow 
management to review patterns or trends regarding mission incidents and 
the effectiveness of the new policy and procedures. 



The Federal Air Marshal 
Service Modified Its Dress 
Code and Hotel Policies to 
Further Protect Air 
Marshals' Anonymity 



To further protect the anonymity of air marshals while on missions, and in 
response to air marshals' feedback and the working groups' 
recommendations, FAMS management revised the dress code policy and 
the hotel policy for air marshals in August 2006 and February 2007, 
respectively. The revisions allow air marshals greater discretion in 
selecting appropriate attire to wear on missions and choosing hotels for 
overnight trips. Before the revisions, air marshals reported that the dress 
code policy was too restrictive and forced them to dress too formally for 
certain flights, such as those to vacation-oriented destinations. According 
to the air marshals, this restrictive policy resulted in their standing out 
from the other passengers, a situation that compromised their anonymity. 
Similarly, before being revised, FAMS's hotel policy directed air marshals 
to stay at certain hotels on overnight missions so that they could be 
located easily by management in an emergency. Additional considerations 
of FAMS management for restricting the hotel selection were to ensure 
that air marshals were able to stay at hotels within per diem rates and 
would have ready access to transportation between the hotel and the 
airport. Air marshals expressed concerns that repeatedly staying at the 
same hotels risked exposing their anonymity. The revised policy allows air 
marshals to select their own hotels, provided the hotels are within per 
diem rates and have adequate transportation options. To alleviate 
concerns of FAMS management about being able to contact air marshals 
in an emergency, the revised policy requires air marshals to report their 
hotel locations via the FAMS intranet. 



All 67 of the air marshals we interviewed in the 11 field offices we visited 
said that the revised dress code and hotel policies adequately addressed 
their concerns. 
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The Federal Air Marshal 
Service Is Planning to 
Replace Its Inventory of 
Personal Digital Assistant 
Communication Devices 
by Early Fiscal Year 2009 



FAMS has described the agency's personal digital assistant (PDA) 
communication device as being a lifeline for air marshals. The current 
device carried by air marshals is intended to function as a cell phone and 
personal computer and allow users to place phone calls, access the 
Internet, send e-mails, pull up basic Microsoft Word documents, store 
documents, and submit reports. However, the findings of FAMS's 
applicable working groups indicated that the current PDA communication 
device has proven unreliable. Similarly, all 67 of the air marshals we 
interviewed in 11 field offices stated that they had experienced problems 
with their PDA device while on missions. Examples of problems reported 
by air marshals included dropped calls or lost signals in certain 
geographical areas, limited audio quality and durability, and lack of ability 
to send certain required documents (such as time and attendance reports). 
Another reported problem was the frequent freezing or locking of the PDA 
device, which then necessitated use of a cumbersome reset process. As a 
result of such problems, air marshals reported that the PDA device has 
hindered their ability to communicate effectively with management while 
in mission status. Additionally, the air marshals we interviewed 
commented that the current PDA device is relatively large and bulky, 
which potentially contributes to loss of anonymity. 

In response to air marshals' feedback and the working groups' 
recommendations, FAMS is taking steps to procure new PDA 
communication devices and distribute them to air marshals. Furthermore, 
according to FAMS officials, the procurement contract for the new PDA 
devices will provide for a 2-year replacement cycle. In the interim, to 
improve voice communication capabilities pending arrival of the new 
devices, FAMS officials reported that it issued new cell phones to air 
marshals in June 2008. The officials noted, however, that air marshals still 
must rely on the current PDA device for non-voice functions, such as 
sending and receiving e-mail messages and documents, until the new PDA 
devices are available. 
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To Address Quality-of-Life 
and Health Issues, the 
Federal Air Marshal 
Service Has Implemented 
Changes to Mission 
Scheduling and Enlisted 
the Assistance of Outside 
Experts 



In reference to quality-of-life and health issues, mission scheduling 
constitutes the most significant concern of air marshals, according to 
feedback that FAMS management received from working groups and other 
communication processes and initiatives. To be fully effective, air 
marshals must be healthy, fit, and alert. However, FAMS's Medical Issues 
Working Group reported that air marshals have experienced various types 
of health issues — poor physical fitness as well as musculoskeletal injuries 
and upper respiratory infections — that may potentially be attributable to 
frequent flying and the overall nature of their jobs. The working group 
noted various challenges to ensuring that air marshals have adequate 
sleep, exercise, and recovery time. A contributing factor noted is that the 
agency's automated scheduling tool historically has lacked the capability 
to consistently program an air marshal's daily start and end times 
throughout a roster period, which makes normal sleep patterns difficult to 
maintain and often results in fatigue. For instance, an air marshal may 
have been scheduled to begin some days at 5 a.m. and other days at 10 
a.m., with unpredictable ending times because of flight delays. In addition 
to inconsistent shifts, the Medical Issues Working Group noted that air 
marshals are subject to long hours — including arriving home late on a 
Friday and then having to depart early the following Monday morning. 
These types of schedules, according to the working group, make allowing 
adequate time for workouts and maintaining healthy eating habits difficult 
and also limit the amount of time available to take care of family and 
personal needs. 



To address these scheduling issues, FAMS has implemented or is planning 
to implement various changes: 



Mission exchange program — This program, which FAMS initially 
piloted in 2006 and is now available to all 21 field offices, allows air 
marshals within the respective field office to exchange mission days 
based on a demonstrated need, such as medical issues or family- 
related issues. For instance, an air marshal with an 8 a.m. mission start 
time and a 9 a.m. medical appointment could exchange shifts with 
another air marshal for a later mission start time. The program is 
intended to reduce the amount of unscheduled leave taken by air 
marshals and otherwise mitigate the hardships or other effects 
associated with FAMS's current policy of requiring air marshals to 
submit requests for annual leave 38 to 66 days in advance. 

Preset ending time and 60-hour rule — In September 2006, FAMS 
instituted a change to its mission-scheduling policy. The change is 
designed to help ensure that air marshals complete their mission 



Page 24 



GAO-09-273 Federal Air Marshal Service 



flights by a preset time on the day before a regular day off (or the day 
before scheduled annual leave) and not begin a new mission until 
receiving a minimum of 60 hours rest. 41 For example, if an air marshal's 
regular days off are Saturday and Sunday, and this individual's mission 
ended on Friday evening, the next mission assignment (on Monday) 
would begin no earlier than Monday morning. 

• Limit on number of flight days — In April 2007, FAMS implemented 
another change in mission-scheduling policy designed to distribute 
flight days equitably and improve the balance between work and 
personal life for air marshals. Specifically, under the new policy, each 
air marshal's total flight days are targeted to not exceed 18 days per 
roster period and 200 days annually. 

• More rest time after completing extended international missions — Also 
in April 2007, FAMS issued guidance to field offices to make every 
attempt at increasing rest time for air marshals after completing an 
extended international mission. Under this guidance, air marshals 
returning from an international mission are to be given a non-flight day 
as their next duty day when any one of the following three conditions 
apply: (1) the return flight exceeds 10 hours in the air, (2) the flight 
crossed the international date line, or (3) the overall mission (round- 
trip flights plus overnight stays) was 4 days or longer in duration. 
Depending on an air marshal's schedule, a non-flight day could be a 
training day, a regular day off, or a non-mission status day. 42 

• More consistent start times — FAMS is currently developing a 
modification to its scheduling tool to provide a consistent, defined 
scheduling window (encompassing, for example, 3 hours) for air 
marshals in mission status to report for duty during a 7-day period. 
Under the planned modification, for instance, FAMS schedulers would 
assign an air marshal to flights departing during 5:00 a.m. to 8:00 a.m., 
9:00 a.m. to 12:00 p.m., or other 3-hour window during the week. FAMS 
officials stated that this modification, which is intended to provide 
more consistent start times for each air marshal throughout the 



A "regular day off refers to each of the 2 days during the 7-day week when an air marshal 
will not be reporting to work. For example, if an air marshal worked a Monday to Friday 
schedule, the regular days off would be Saturday and Sunday. 

42 According to FAMS workday rules, an air marshal in "non-mission status" is at work 
performing administrative or other tasks and is not expected to fly while in this status. 
Under workday rules, air marshals generally have 11 days in non-mission status each year. 
In addition, air marshals are not expected to fly during training days (20 days annually). 
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applicable week, should be completed and ready for pilot testing by 
the middle of calendar year 2009. 

The 67 air marshals we interviewed in 11 field offices generally expressed 
satisfaction with the various enhancements to mission scheduling, 
although most (43) mentioned that implementation of the mission 
exchange program was still evolving. 

To more specifically address the health implications of flight scheduling, 
several efforts were recently completed or are planned. For instance, the 
Homeland Security Institute conducted a benchmark analysis and 
assessment of fatigue issues related to air marshals and issued a report to 
FAMS in July 2008. 43 In its analysis, the institute compared FAMS's 
workday rules against other occupations — largely in the aviation realm — 
that face challenges involving frequent travel, jet lag, long work hours, 
rotating shifts, and the stress of maintaining a schedule across multiple 
flights and airports. The Homeland Security Institute noted that although 
no other occupation is identical to that of air marshals, meaningful 
comparisons were made with similar occupations, such as commercial 
airline pilots and cargo pilots and law enforcement officers working in 
aviation (e.g., U.S. Marshals Service aviation enforcement officers 
responsible for transporting prisoners). In its July 2008 report, the 
Homeland Security Institute noted that while stress and fatigue issues are 
a part of all organizations and cannot be entirely eliminated, air marshals 
are provided considerable blocks of rest within their schedules, when 
assessed against similar occupations. Overall, the institute reported that 
the results of the benchmark analysis showed that air marshals are 
provided above-average time to recuperate from duty days. 

Further, the institute noted that FAMS has taken various steps, including 
implementation of the mission exchange program, to improve aspects of 
mission scheduling. In addition, in October 2008, FAMS officials informed 
us that the agency has funded a contract with the National Institute of 
Justice to implement FAMS-specific research regarding mission 
scheduling, work-rest cycles, fatigue, and performance. 



Homeland Security Institute, Federal Air Marshal Service (FAMS) Risk-Based Resource 
Allocation Study: Human Factors Analysis, final report (July 10, 2008). 
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To Further Address 
Quality-of-Life Issues, the 
Federal Air Marshal 
Service Has Implemented a 
Voluntary Lateral Transfer 
Program 



According to FAMS officials, air marshals frequently cited the need for a 
voluntary lateral transfer program during listening group sessions and 
dinners with the director. FAMS working groups that examined quality-of- 
life issues also reported that the agency would benefit from implementing 
a transfer program for air marshals to express interest in relocating to 
another field office. Thus, in October 2006, FAMS management 
implemented a voluntary lateral transfer program. Under the program, an 
air marshal in good standing may request a transfer for up to three field 
offices, ranked by order of preference, and FAMS management is to make 
decisions based on the number of vacancies in each office and the 
seniority of air marshals who apply for transfer. 



In December 2006, FAMS announced that 176 air marshals had been 
selected, during the first phase of the program, for transfer to new offices 
within 60 days. In the second phase, which occurred in the spring of 2007, 
FAMS management made transfer offers to 40 air marshals — all of whom 
accepted. In the third and most recent phase, which occurred in the spring 
of 2008, FAMS management made offers to 48 air marshals — of whom 45 
accepted. FAMS expects to continue offering voluntary transfer 
opportunities during open seasons in the spring of each year. 



The Federal Air Marshal 
Service Conducted a 
Workforce Satisfaction 
Survey in 2007; the 
Potential Usefulness of 
Future Surveys Could Be 
Enhanced by Improving 
the Response Rate and 
Redesigning Certain 
Questions 



In late fiscal year 2007, FAMS conducted a workforce satisfaction survey 
of all staff — not just air marshals — to help determine issues affecting the 
ability of agency personnel to perform their jobs — and to obtain feedback 
on the effectiveness of measures already taken by management to address 
relevant issues. 44 The 2007 survey questionnaire consisted of a total of 60 
questions that covered 13 topics — senior leadership; 
supervisor/management; resources and technology; training and 
education; career development; policies and procedures; employee 
involvement and autonomy; rewards and recognition; communication; 
safety, health, and medical issues; work and family life; organizational 
commitment; and job satisfaction. 45 According to FAMS management 
officials, the survey provided useful information on quality-of-life and 



FAMS has conducted two other surveys of employees. One of these was conducted 
earlier in 2007 and focused on communications and operational concepts. The other 
survey, conducted in 2006, focused on quality-of-life issues. 

45 See appendix VI for a copy of the survey questionnaire. In addition to the 60 substantive 
questions, the survey questionnaire also had 10 questions that solicited demographic 
information from the respondents. 
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other issues affecting the ability of air marshals and other agency 
personnel to perform their jobs. In addition, the officials reported that 
survey results indicated that employees generally were pleased with the 
policy changes and other actions implemented by management to address 
relevant issues. For example, although the 2007 workforce satisfaction 
survey had an overall response rate (46 percent) that constituted less than 
half of the FAMS workforce, 79 percent of the respondents indicated that 
there had been positive changes from the prior year. Regarding future 
plans, FAMS expects to administer a workforce satisfaction survey every 2 
years. FAMS officials stated that a purpose of the initial workforce 
satisfaction survey was to establish a baseline for use in comparing the 
results of future surveys. 

In reviewing the 2007 survey's implementation and results, we made 
several observations that are important for enhancing the potential 
usefulness of future surveys. First, as noted previously, the overall 
response rate was 46 percent. FAMS officials expressed satisfaction with 
this response rate given the highly mobile nature of their workforce. The 
FAMS officials also noted that the 46 percent response rate was similar to 
the response rates for other federal workforce satisfaction surveys. 
However, the 46 percent response rate was substantially less than the 80 
percent rate OMB encourages for federal surveys that require its approval. 
Although internal workforce surveys such as the one conducted by FAMS 
do not require OMB approval, we believe the OMB standards and guidance 
provide relevant direction on planning, designing, and implementing high- 
quality surveys — including the need to obtain a high response rate to 
increase the potential that survey responses will accurately represent the 
views of the survey population. 46 

Specifically, the OMB guidance stipulates that agencies must design 
surveys to achieve the highest practical rates of response to ensure that 
the results are representative of the target population and that they can be 
used with confidence as input for informed decision-making. OMB 
encourages agencies to obtain at least an 80 percent response rate, and its 
guidance states that response rates are an important indicator of the 
potential for nonresponse bias, which could affect the accuracy of a 
survey's results. For instance, survey estimates may be biased if the 



The OMB guidance governs federal agency surveys of the public at large or outside 
individuals, groups, or organizations, such as local government entities. The FAMS 
workforce survey was administered internally to gather information from the agency's 
employees. 
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individuals who choose to participate (respondents) differ substantially 
and systematically in some way from those who choose not to participate 
(nonrespondents). In general, a higher response rate increases the 
likelihood that any bias problem is decreased, resulting in the views and 
characteristics of the target population being more accurately reflected in 
the survey's results. Thus, for any federal survey that must be approved by 
OMB, applicable guidelines stipulate that an analysis for possible 
nonresponse bias must be conducted if the final response rate is less than 
80 percent. 47 

Regarding the 46 percent response rate for the 2007 survey, FAMS 
management reported that an analysis of potential nonresponse bias was 
conducted by comparing various demographic data provided by the 
respondents to the FAMS workforce as a whole. Based on the analysis of 
the available demographic data, FAMS concluded that nonresponse bias 
did not exist as the respondents were representative of the entire 
workforce. Although the analysis conducted by FAMS was a useful effort, 
the potential for a nonresponse bias still exists given that over half of the 
FAMS workforce did not respond to the survey. 

As noted previously, concerns about nonresponse bias could be avoided or 
mitigated by obtaining a higher response rate. FAMS employees were 
given 3 weeks (August 23 through September 14, 2007) to complete the 
2007 workforce satisfaction survey. According to FAMS management, even 
though all employees (not just nonrespondents) were sent four messages 
reminding them of the deadline for completing the voluntary survey, the 
final overall response rate was 46 percent. We believe that other widely 
acknowledged methods, outlined in OMB guidance, could improve the 
response rate of future FAMS surveys. These methods include, for 
example, promoting awareness of the survey through outreach efforts with 
groups of prospective respondents and extending the cut-off date for 
responding to the survey. Also, monitoring questionnaire returns and 
targeting extra follow-up efforts to air marshals in particular field 
locations that have comparatively low response levels could help. 

Additional observations we made in reviewing the 2007 workforce 
satisfaction survey's questionnaire involve the sentence structure of 



Office of Management and Budget, Standards and Guidelines for Statistical Surveys 
(Washington, D.C.: September 2006), p. 16; and, Office of Management and Budget, Office 
of Information and Regulatory Affairs, Questions and Answers When Designing Surveys 
for Information Collections (Washington, D.C.: Jan. 20, 2006), p. 56. 
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certain questions and the response options. Generally, any question that 
combines two or more issues — but does not provide for separate or 
respective answers — can cause uncertainty about how to respond if the 
answer to each issue is different. 48 Table 2 lists the seven 2007 workforce 
satisfaction survey questions that used these types of sentence structures. 
For instance, regarding the senior leadership of FAMS, question 3 cites 
two concepts ("visions" and "initiatives"), as well as two actions ("shared" 
and "supported") associated with these concepts. However, the response 
options did not account for the fact that experiences could be different 
with each of these concepts and actions. Similarly, question 10 addresses 
the reliability of equipment used by agency personnel and cites four 
different devices. However, the response options did not account for the 
fact that experiences could be different with each of these devices. 



Table 2: FAMS 2007 Workforce Satisfaction Survey Questions That Combined Two 
or More Issues 


Survey question topic, number, and wording 


Topic 


Number 


Wording 


Senior Leadership 


3 


I believe that the Director's visions and 
initiatives are consistently shared and 
supported by my field management. 


Resources and technology 


10 


Generally, the equipment I use (e.g., firearm, 
computer, cell phone, personal digital 
assistant, etc.) to perform my job works 
properly. 


Training and education 


16 


Generally, I am satisfied with the content and 
variety of job-related training I receive in my 
office. 


Communication 


36 


FAMS policies and procedures are clearly 
communicated and easy to understand." 


Safety, health, and 
medical issues 


43 


I am satisfied that FAMS management is 
concerned for the health and safety of 
employees and is working continuously to 
offer improved services. 




44 


I have been provided information and 
resources to take personal responsibility for 
my health and wellness as it relates to my job 
(e.g., proper diet, fitness, sufficient rest). 



Specifically, a question containing the word "and" raises a concern about whether the 
respective portions of the question on either side of the conjunction are sufficiently related 
to be considered similar or are really different topics or concepts. Such questions are called 
"double barreled" within the social science community. 
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Survey question topic, number, and wording 



Topic 


Number 


Wording 


Work and family life 


51 


I have seen improvement in quality of work 






life and family life as a result of the 






recommendations from the Director's working 






groups. 


Source: FAMS 2007 workforce satisfaction survey. 

Note: See appendix VI. 



Also, none of the 60 questions in the 2007 workforce satisfaction survey 
provided for response options such as "not applicable" or "no basis to 
judge" — responses that would be appropriate when respondents had little 
or no familiarity with the topic in question. Not providing response options 
such as "not applicable" or "no basis to judge" could lead to potentially 
misleading question responses. In the interest of being compliant, 
respondents might be compelled to give a response, such as "neutral," to a 
question when they actually have no opinion due to either non- 
applicability or lack of familiarity with the question topic. While it might 
be assumed that all individuals being surveyed should be familiar with the 
topic of all questions, this might not be the case and will not be known 
unless the questionnaire contains the relevant response options. For 
example, question 39 (see app. VI), reads as follows: "I am satisfied that 
the work-related concerns I address with management are addressed 
appropriately." As written, this sentence assumes that every employee has 
raised work-related concerns with management. If a respondent had never 
expressed work-related concerns with management, this individual might 
not know how to respond, given the question's existing response options. 

Thus, based on the sentence structure of certain questions and the 
response options, the results from the 2007 survey may provide an 
incomplete assessment of employees' perspectives and attitudes to FAMS 
management. Regarding our observations on the design of survey 
questions and response options, FAMS officials stated that limited 
personnel resources precluded investing more time in development of the 
survey questionnaire and that the survey had served a useful purpose in 
providing information on issues of concern to be more fully explored 
through other communication processes or initiatives. Nonetheless, in 
developing future survey instruments, designing questions to avoid these 
types of ambiguities could provide FAMS management with information 
that is more focused and complete. Although we recognize that FAMS has 
a variety of other processes and initiatives — in addition to the customer 
satisfaction survey — for identifying and addressing workforce issues, 
customer satisfaction surveys can be particularly useful given that they are 
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distributed to all employees and provide for anonymity of respondents. 
Further, the design considerations that we discussed involve relatively 
minor technical aspects that could be addressed with a minimal 
investment of personnel resources. 



The Federal Air Marshal 
Service Has Committed to 
Addressing Workforce 
Issues by Sustaining 
Applicable Processes and 
Initiatives 



As highlighted in our prior work, agency leaders in best practice 
organizations view people as an important enabler of agency performance 
and recognize the need for sustained commitment to strategically manage 
human capital. 49 In developing approaches to managing the workforce, 
leaders of best practice agencies seek out the views of employees at all 
levels. Involving employees in the planning process helps agencies to 
develop goals and objectives that incorporate frontline insights and 
perspectives about operations. Further, such involvement can also serve 
to increase employees' understanding and acceptance of organization 
goals and objectives and improve motivation and morale. Our work has 
shown that leading organizations commonly sought their employees' input 
on a periodic basis and used that input to adjust their human capital 
approaches. Among other means, the organizations collected feedback by 
convening focus groups, providing opportunities for employees to 
participate in working groups or task forces, and conducting employee 
satisfaction surveys. 



As discussed earlier in this report, FAMS has implemented a variety of 
processes and initiatives to address workforce issues by soliciting the 
views of front-line staff across the agency. Several key improvements in 
FAMS policies and procedures have resulted from these efforts. Among 
other improvements, for example, FAMS amended its policy for flight 
check-in and boarding procedures to better ensure the anonymity of air 
marshals in mission status. Also, the various processes and initiatives have 
helped to improve agency morale, according to the federal air marshals we 
interviewed. Moreover, agency officials noted that the processes and 
initiatives represented a significant commitment in management time and 
resources. 

In our view, fostering continued progress in addressing workforce issues 
at FAMS is important. The current FAMS Director, after being designated 
in June 2008 to head the agency, issued a broadcast message to all 



GAO, A Model of Strategic Capital Management, GAO-02-373SP (Washington, D.C.: 
March 2002). 
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employees, expressing a commitment to continue applicable processes 
and initiatives, including the working group process, listening sessions, 
field office visits, and the internal Web site for agency personnel to 
provide anonymous feedback to management on any topic. More recently, 
in response to our inquiry, FAMS's Chief of Staff reported in October 2008 
that the various communications processes and initiatives "have become 
an institutionalized and positive aspect" of the agency's culture. Also, the 
Chief of Staff noted that FAMS was in the process of establishing an 
agencywide national advisory council — with representatives from 
headquarters and all field offices — to further enhance communication and 
outreach efforts, promote greater job satisfaction, and improve 
organizational effectiveness through cooperative problem solving and 
replication of best practices. 



Federal air marshals are an important layer of aviation security. Thus, it is 
incumbent upon FAMS management to have sound management processes 
in place for identifying and addressing the challenges associated with 
sustaining the agency's operations and addressing workforce quality-of-life 
issues. FAMS, to its credit, has established a number of processes and 
initiatives — including a workforce satisfaction survey — to address various 
operational and quality-of-life issues that affect the ability of air marshals 
and other FAMS personnel to perform their aviation-security mission. 
Consistent with the human capital practices of leading organizations, the 
current FAMS Director has expressed a commitment to continuing 
relevant processes and initiatives for identifying and addressing workforce 
concerns, maintaining open lines of communications, and sustaining 
forward progress. Although the workforce satisfaction survey is only one 
of a number of processes or initiatives used by FAMS to identify and 
address workforce issues, such surveys play an important role given their 
agencywide scope and the provision for anonymous responses. A higher 
response rate and more clearly structured questions and response options 
could add to the usefulness of this effort. 



To facilitate continued progress in identifying and addressing issues that 
affect the ability of FAMS personnel to perform the agency's aviation- 
security mission, we recommend that the FAMS Director take appropriate 
actions to increase the usefulness of the workforce satisfaction surveys 
that FAMS plans to conduct biennially. Such actions could include, for 
example, ensuring that the survey questions and the answer options are 
clearly structured and unambiguous and that additional efforts are 
considered for obtaining the highest possible response rates. 



Conclusions 



Recommendation for 
Executive Action 
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We provided a draft of our restricted report for comment to the 
Department of Homeland Security and TSA. In November 2008, in written 
comments, the Department of Homeland Security and TSA agreed with 
our recommendation and noted that FAMS was in the initial stages of 
formulating the next workforce satisfaction survey, which included plans 
to implement the recommendation. Also, the Department of Homeland 
Security and TSA commented that our key findings and recommendation 
will facilitate continued progress in identifying and addressing issues that 
affect the ability of FAMS personnel to perform the agency's aviation 
security mission. The full text of the department's and TSA's written 
comments is reprinted in appendix VII. 



As arranged with your office, we plan no further distribution until 30 days 
after the date of this report. At that time, we will send copies of this report 
to interested congressional committees and subcommittees. We will also 
make copies available to others upon request. 

If you or your staff have any questions about this report or wish to discuss 
the matter further, please contact me at (202) 512-4379 or lords@gao.gov. 
Contact points for our Offices of Congressional Relations and Public 
Affairs may be found on the last page of this report. Other key contributors 
to this report are listed in appendix VIII. 




Stephen M. Lord 

Acting Director, Homeland Security and Justice Issues 



Agency Comments 
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List of Congressional Requesters 

The Honorable Susan M. Collins 
Ranking Member 

Committee on Homeland Security and Governmental Affairs 
United States Senate 

The Honorable Daniel K. Akaka 
United States Senate 

The Honorable Bennie G. Thompson 
Chairman 

Committee on Homeland Security 
House of Representatives 

The Honorable Sheila Jackson-Lee 
House of Representatives 

The Honorable Christopher P. Carney 
House of Representatives 



Page 35 



GAO-09-273 Federal Air Marshal Service 



Appendix I: Objectives, Scope, and 
Methodology 



ObjGCtiVGS ^ a * s re P or *' addresses the following three principal questions: 



What is the Federal Air Marshal Service's operational approach for 
achieving its core mission of providing an onboard security presence 
for flights operated by U.S. commercial passenger air carriers? 



• To what extent has the Federal Air Marshal Service's operational 
approach for achieving its core mission been independently assessed? 



• To what extent does the Federal Air Marshal Service have processes 
and initiatives in place to address issues that affect the ability of its 
workforce to carry out its mission? 



SCODG cUld Initially, to obtain contextual and overview perspectives regarding the 

" principal questions, we reviewed information available on the Web sites of 

Methodology relevant federal entities — the Department of Homeland Security, the 

Transportation Security Administration (TSA), and the Federal Air 
Marshal Service (FAMS). To obtain additional perspectives regarding 
FAMS's mission and operations — and issues affecting its workforce — we 
conducted a literature search to identify relevant reports or studies and 
other publicly available information, including news media articles. In 
particular, we focused on reviewing congressional studies, Inspector 
General reports, and our previous reports. These included the following: 



• U.S. House of Representatives, Committee on the Judiciary, Plane 
Clothes: Lack of Anonymity at the Federal Air Marshal Service 
Compromises Aviation and National Security (Washington, D.C.: 
May 25, 2006). 



• GAO, Aviation Security: Federal Air Marshal Service Could Benefit 
from Improved Planning and Controls, GAO-05-884SU (Washington, 
D.C.: Sept. 29, 2005). The report is restricted (not available to the 
public) because it contains sensitive security information. The public 
version of the report is GAO-06-203 (Nov. 28, 2005). 



• U.S. Department of Homeland Security, Office of Inspector General, 

Review of Alleged Actions by Transportation Security 
Administration to Discipline Federal Air Marshals for Talking to the 
Press, Congress, or the Public, OIG-05-01 (Washington, D.C.: Nov. 
2004). 
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Methodology 



• GAO, Budget Issues: Reprogramming of Federal Air Marshal Service 
Funds in Fiscal Year 2003, GAO-04-577R (Washington, D.C.: Mar. 31, 
2004). 

• GAO, Aviation Security: Federal Air Marshal Service Is Addressing 
Challenges of Its Expanded Mission and Workforce, but Additional 
Actions Needed, GAO-04-242 (Washington, D.C.: Nov. 19, 2003). 

This report is the public version of a restricted report that we provided to 
congressional requesters in December 2008. 1 Further details about the 
scope and methodology of our work regarding each of the three principal 
questions are presented in the following sections, respectively. 



In addressing this topic, we reviewed relevant legislation regarding 
FAMS's mission and organizational structure. In particular, we reviewed a 
provision of the Aviation and Transportation Security Act that 

• requires the deployment of federal air marshals on passenger airline 
flights and 

• specifically requires the deployment of federal air marshals on every 
flight determined to present high security risks. 2 

We analyzed FAMS documentation regarding the agency's strategy and 
concept of operations for carrying out its mission. Also, we reviewed the 
results of an evaluation conducted in 2003 by the Office of Management 
and Budget (OMB), which utilized its Program Assessment Rating Tool 
(PART) to assess the management and performance of FAMS and 
concluded that key aspects of program design needed to be independently 
assessed. 3 Further, we reviewed the follow-on PART-related reassessment 
of FAMS that OMB conducted in 2008 (see app. II). 



1 GAO, Aviation Security: Federal Air Marshal Service Has Taken Actions to Fulfill Its 
Core Mission and Address Workforce Issues, but Additional Actions Are Needed to 
Improve Workforce Survey, GAO-09-53SU (Washington, D.C.: Dec. 8, 2008). 

2 Pub. L. No. 107-71, § 105, 115 Stat. 597, 606-08 (2001) (codified as amended 49 U.S.C. § 
44917). 

3 PART consists of a standard series of questions intended to determine the strengths and 
weaknesses of federal programs. The PART questions cover four broad topics — (1) 
program purpose and design, (2) strategic planning, (3) program management, and (4) 
program results/accountability. See GAO, Program Evaluation: OMB's PART Reviews 
Increased Agencies' Attention to Improving Evidence of Program Results, GAO-06-67 
(Washington, D.C.: Oct. 28, 2005). 



The Federal Air Marshal 
Service's Operational 
Approach for Achieving Its 
Core Mission 
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Independent Assessment 
of the Federal Air Marshal 
Service's Operational 
Approach 



We reviewed the July 2006 classified report prepared by the Homeland 
Security Institute based on its independent evaluation of FAMS's concept 
of operations. 4 Our engagement team included a social science analyst and 
an economist with experience in risk assessment, who used generally 
accepted social science research standards in reviewing the institute's 
report. Also, we interviewed applicable Homeland Security Institute 
officials to enhance our understanding of the evaluation's scope, 
methodology, findings, and recommendations. Based on our review and 
discussion, we determined this report to be sufficiently reliable for the 
purposes of our work. Further, we reviewed FAMS documentation — and 
interviewed the Director of FAMS and other senior officials at the agency's 
headquarters — regarding the status of efforts to address recommendations 
made by the Homeland Security Institute and any related initiatives 
involving strategic planning and the agency's concept of operations. 5 

We also reviewed two additional Homeland Security Institute reports, 
which FAMS provided to us in September 2008. One of the reports detailed 
the institute's analysis regarding requirements for a next-generation 
mission scheduling tool for FAMS, 6 and the other report presented the 
institute's benchmark analysis that compared FAMS's workday rules 7 and 
practices against those of similar occupations involving frequent air travel 



4 The Homeland Security Institute is a federally funded research and development center 
established pursuant to the Homeland Security Act of 2002. See Pub. L. No. 107-296, § 312, 
116 Stat. 2135, 2176, as amended. The institute's mission is to assist the Department of 
Homeland Security address relevant issues requiring scientific, technical, and analytical 
expertise. In response to OMB's 2003 PART review of FAMS, which concluded that key 
aspects of program design needed to be independently assessed, FAMS contracted for an 
independent evaluation to be conducted by the Homeland Security Institute. 

5 Appendix III presents quoted excerpts that substantially replicate the executive summary 
in the Homeland Security Institute's July 2006 report. Appendix IV presents an overview of 
performance measures for assessing FAMS. 

6 Homeland Security Institute, Analysis for the Next-Generation Federal Air Marshal 
Service (FAMS) Mission Scheduling & Notification System (MSNS), final report (July 10, 
2008). 

"Workday rules" refer to the parameters that FAMS uses for assigning air marshals to 
flights. As applicable to non-overnight missions, for example, FAMS tries to assign air 
marshals to flights (or combinations of flights) that will return the air marshals back home 
during a scheduled 10-hour workday. 
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and the related operational challenges, including fatigue and other human 
factors. 8 



The Federal Air Marshal 
Service's Processes and 
Initiatives for Addressing 
Issues That Affect Its 
Workforce 



Regarding operational or tactical issues as well as quality-of-life issues that 
affect the ability of air marshals to carry out the agency's mission, we 
reviewed published reports, including our September 2005 report 
(GAO-05-884SU) as well as news media accounts of relevant issues. We 
also reviewed FAMS documentation regarding various working groups 
(see app. V) and other initiatives that FAMS had established to address 
issues that affect the ability of air marshals to carry out the agency's 
mission. In particular, we reviewed the final report (if available) produced 
by the respective working group. For criteria in reviewing the agency's 
documentation regarding these efforts, we drew on our prior work 
regarding leading organizations and the best practices for strategically 
managing human capital. 9 



Further, we interviewed the Director of FAMS and other senior officials at 
agency headquarters, and we visited 11 of the agency's 21 field offices, 
where we interviewed managers and a total of 67 air marshals. 10 We 
selected the 11 field offices and the 67 air marshals based on 
nonprobability sampling. 11 In selecting the 11 field offices, we considered 
various factors, such as geographic location of the offices and the 
involvement of local management in agencywide working groups to 
address issues affecting air marshals. 

At each of the 11 field offices, we first reviewed available work-related 
information about individual air marshals, such as their starting dates with 
FAMS and their involvement in ground-based assignments or any 



Homeland Security Institute, Federal Air Marshal Service (FAMS) Risk-Based Resource 
Allocation Study: Human Factors Analysis, final report (July 10, 2008). 

9 See GAO, A Model of Strategic Human Capital Management, GAO-02-373SP 
(Washington, D.C.: Mar. 15, 2002). 

10 The field offices we visited are not listed in this report because TSA considers field office 
locations to be Sensitive Security Information. 

11 Nonprobability sampling is a method of sampling where observations are selected in a 
manner that is not completely random, generally using specific characteristics of the 
population as criteria. Results from a nonprobability sample cannot be used to make 
inferences about an entire population because some elements of the population being 
studied had no chance or an unknown chance of being selected as part of the sample. 



Page 39 



GAO-09-273 Federal Air Marshal Service 



Appendix I: Objectives, Scope, and 
Methodology 



agencywide working groups. 12 Based on these factors, we selected and 
interviewed 6 to 7 air marshals at each of the 11 field offices. Specifically, 
we selected 6 air marshals at each of 10 field offices and 7 air marshals at 
the remaining office. Our selections were made to encompass a variety of 
experience levels. Also, at each field office, rather than meeting separately 
with each individual, we conducted the interviews of the selected air 
marshals in group settings to encourage a wide array of perspectives, 
whether corroborating or contradictory. We conducted our interviews at 
the field offices during a 7-month time period, July 2007 through January 
2008. 

Because we selected a nonprobability sample of FAMS field offices to visit 
and air marshals to interview, the information we obtained in these visits 
and interviews cannot be generalized either to all 21 field locations or to 
all air marshals in the offices we visited. However, the visits and 
interviews provided us a broad overview of issues important to air 
marshals. 

We reviewed documentation regarding the implementation and results of a 
workforce satisfaction survey that FAMS conducted in 2007. Our 
engagement team, which included social science analysts with extensive 
survey research experience, reviewed the questionnaire used in the survey 
for clarity and the related response options for appropriateness (see app. 
VI). Also, we discussed with FAMS officials the extent to which efforts 
were made to obtain an overall response rate as high as possible. As 
criteria to guide our review of the survey results, we used the following 
OMB guidance: 

• Standards and Guidelines for Statistical Surveys (September 2006). 

• Questions and Answers When Designing Surveys for Information 
Collections (Jan. 20, 2006). 

We conducted this performance audit from April 2007 to December 2008 in 
accordance with generally accepted government auditing standards. Those 
standards require that we plan and perform the audit to obtain sufficient, 
appropriate evidence to provide a reasonable basis for our findings and 
conclusions based on our audit objectives. We believe that the evidence 



Examples of ground-based assignments for air marshals included work in a field office's 
operations or training sections or in an airport liaison position. These assignments are 
temporary with varying lengths of service. 
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obtained provides a reasonable basis for our findings and conclusions 
based on our audit objectives. 
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The Office of Management and Budget's (OMB) Program Assessment 
Rating Tool (PART) consists of a standard series of 25 questions intended 
to determine the strengths and weaknesses of federal programs. The 25 
total questions cover 4 broad topics — (1) program purpose and design, (2) 
strategic planning, (3) program management, and (4) program 
results/accountability. 

This appendix provides an overview of OMB's PART-based assessments of 
the Federal Air Marshal Service (FAMS) conducted in 2003 and 2008. 
Additionally, for each of the 25 questions used in the PART-based 
assessments, the appendix compares OMB's 2003 and 2008 answers and 
summarizes OMB's narrative findings (explanation and evidence). Also, 
when the answers in 2003 and 2008 differed for a particular question, the 
appendix briefly explains the basis for the respective answer. 

More detailed information regarding the 2003 and 2008 OMB PART 
assessments of FAMS can be found on OMB's Web site: 
www.ExpectMore.gov. 



OMB's PART assessments of federal agencies provide performance ratings 
that indicate how effectively tax dollars are spent. Following an 
assessment, OMB assigns an agency one of five possible overall ratings: 

• Effective. Programs rated "effective" set ambitious goals, achieve 
results, are well-managed, and improve efficiency. 

• Moderately effective. A "moderately effective" rating indicates a 
program that sets ambitious goals and is well-managed but needs to 
improve its efficiency or address other problems in the programs' 
design or management in order to achieve better results. 

• Adequate. An "adequate" rating describes a program that needs to set 
more ambitious goals, achieve better results, improve accountability, 
or strengthen its management practices. 

• Ineffective. An "ineffective" rating indicates a program that fails to use 
tax dollars effectively and is unable to achieve results because of a 
lack of clarity regarding the program's purpose or goals, poor 
management, or some other significant weakness. 

• Results not demonstrated. A "results not demonstrated" rating 
indicates that a program has been unable to develop acceptable 
performance goals or collect data to determine whether it is 
performing. 



Overview of the 2003 and 
2008 Assessments of the 
Federal Air Marshal 
Service 
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In OMB's 2003 PART assessment, FAMS received a rating of "results not 
demonstrated" because at that time FAMS did not have measurable 
results. Additionally, OMB cited strategic planning deficiencies that 
included the absence of baselines, targets, and time frames associated 
with performance goals and performance measurements. OMB further 
noted the absence of a second long-term outcome measure, 1 proxy 
measures with respect to deterrence, and an efficiency measure. 

In OMB's 2008 PART assessment, FAMS received a rating of "moderately 
effective." Regarding the improved rating, OMB recognized the Homeland 
Security Institute's independent evaluation, which endorsed FAMS's 
concept of operations. Also, OMB noted that FAMS had addressed other 
deficiencies by developing the following performance measures: 2 

• A second long-term outcome measure — the level of public confidence 
in air marshals' ability to promote aviation security — which is 
reflective of FAMS's purpose. 

• Proxy measures of deterrence, such as air marshals' average annual 
rate of accuracy in firearms requalification testing. 

• Efficiency measures, such as (a) cost per flight per air marshal and (b) 
percentage of air marshals meeting the targeted number of flying days 
per year. 



The results of PART'S 25 questions in reference to the 2003 and 2008 
assessments of FAMS are presented in tables 3 through 6, specifically: 

• The 5 questions in table 3 cover program purpose and design. 

• The 8 questions in table 4 cover strategic planning. 

• The 7 questions in table 5 cover program management. 

• The 5 questions in table 6 cover program results/accountability. 

As presented in tables 3 through 6, the narrative discussion (explanation 
and evidence) is our summary of OMB's key points. If needed for purposes 
of clarifying the respective topic or ensuring accuracy, we used additional 
or alternative wording to summarize OMB's findings. Also, in a few 
instances, we updated the information as appropriate. 



Section-by-Section Details 
of the 2003 and 2008 
Assessments of the 
Federal Air Marshal 
Service 



1 At the time of OMB's 2003 PART assessment, FAMS had one long-term outcome measure: 
"Number of successful terrorist and other criminal attacks initiated from commercial 
passenger aircraft cabins with air marshal coverage." 

2 Appendix IV presents more detailed information about FAMS's performance measures. 
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Answer 

Section 1 questions, explanation, and evidence 2003 2008 



Question 1.1: Is the program purpose clear? 



Appendix II: OMB's Program Assessment 
Rating Tool Results in 2003 and 2008 for the 
Federal Air Marshal Service 



Yes Yes 



Explanation and Evidence: The 2008 assessment noted that FAMS's purpose is to promote confidence in the 
nation's civil aviation system through the effective deployment of air marshals to detect, deter, and defeat hostile 
acts targeting U.S. air carriers, airports, passengers, and crews. Also, the assessment noted the following: 

49 U.S.C. § 1 1 4(q) a authorizes the Secretary of Homeland Security — and by delegation, the Administrator of 
the Transportation Security Administration (TSA) — to designate employees of TSA as federal law 
enforcement officers. The authority provides that these officers may carry a firearm, make arrests, and seek 
and execute warrants for arrest or seizure of evidence. 

Section 1303 of the Implementing Recommendations of the 9/1 1 Commission Act of 2007 amended the 
Homeland Security Act of 2002" to authorize the Secretary of the Department of Homeland Security to 
develop Visible Intermodal Prevention and Response teams for augmenting the security of any mode of 
transportation at any U.S. location. 



Question 1 .2: Does the program address a specific and existing problem, interest, or need? Yes Yes 

Explanation and Evidence: The 2008 assessment stated that President Bush ordered the rapid expansion of 
FAMS as a result of the September 1 1th attacks in order to restore public confidence in air travel and prevent 
further attacks. Also, the assessment cited the following: 

In establishing the Department of Homeland Security (DHS), the Homeland Security Act of 2002 noted that 
FAMS is critical to aviation security. 

The National Strategy for Aviation Security categorizes protection of the air domain as a vital national interest 
and cites federal air marshal presence on commercial passenger aircraft as among the efforts responsible for 
enhancing aviation security on an ongoing basis. 

A message to DHS employees from Secretary Chertoff, dated August 10, 2006, stated that FAMS would 
expand mission coverage for U.K.-U.S. flights as a component of the U.S. government's response to the 
Heathrow plot" and DHS's responsibility to protect travelers. 



Question 1 .3: Is FAMS designed so that it is not redundant or duplicative of any federal, state, local, or private Yes No 
effort? 

Explanation and Evidence: Regarding the "No" answer, the 2008 assessment indicated that the design of aviation 
security includes layers of redundancies and duplicate security measures to reinforce each and every layer for the 
strongest level of security. However, the assessment also noted the following: 

As a critical layer of aviation security, FAMS fills an important role in detecting, deterring, and defeating 
criminal terrorist activities on aircraft. For international flights, only air marshals are authorized to fly armed. 

Although the presence of other armed law enforcement officers and Federal Flight Deck Officers 8 on 
domestic flights may at times augment air marshals, the distinctive differences inherent in the FAMS program 
prevent redundancy or excessive overlap. 
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Answer 


Section 1 questions, explanation, and evidence 


2003 2008 



Question 1 .4: Is the program design free of major flaws that would limit the program's effectiveness or efficiency? No Yes 

Explanation and Evidence: According to the 2003 assessment, key aspects of the program design needed to be 
independently assessed and validated. The 2008 assessment recognized that the Homeland Security Institute 
conducted an independent evaluation (document is classified) of the development and implementation of FAMS's 
concept of operations in 2006. Given FAMS's objectives and performance goals, the institute found that FAMS 
applies a valid approach to analyzing risk and allocating resources on the basis of threat, vulnerability, and 
consequence. 



Question 1 .5: Is the program effectively targeted, so program resources reach intended beneficiaries and/or NA 1 Yes 

otherwise address the program's purpose directly? 

Explanation and Evidence: According to the 2003 assessment, FAMS considered the entire flying public to be the 
intended beneficiaries of the air marshal security program. According to the 2008 assessment, the application of 
FAMS's concept of operations effectively targeted resources by prioritizing and scheduling missions on the basis 
of risk. The assessment noted that FAMS structures deployments so that program benefits reach highest risk 
beneficiaries at the micro level, while deterrence as a macro-level benefit reaches the flying public (collectively) 
and other intended beneficiaries. 



Source: OMB's 2003 and 2008 PART reviews of FAMS. 

"The provision cited, which authorizes the Under Secretary of Transportation for Security (now the 
Assistant Secretary, TSA) to designate employees of TSA or other federal agencies to serve as law 
enforcement officers, has subsequently been redesignated §1 14(p). 

"Section 1303 does not specifically amend the Homeland Security Act but does provide specific 
authority to the Secretary, acting through the Assistant Secretary, TSA, to establish Visible Intermodal 
Prevention and Response teams. 

c After the March 2004 train bombings in Madrid, TSA created and deployed Visible Intermodal 
Prevention and Response teams to enhance security on U.S. rail and mass transit systems 
nationwide. Comprised of federal air marshals, surface transportation security inspectors, 
transportation security officers, behavior detection officers, and explosives detection canines, the 
teams are intended to work with local security and law enforcement officials to supplement existing 
security resources, provide a deterrent presence and detection capabilities, and introduce an element 
of unpredictability to disrupt potential terrorist activities. 

"The Heathrow plot was a terrorist plan to blow up 10 commercial airplanes bound for the United 
States from the United Kingdom. On August 10, 2006, British law enforcement arrested 24 suspects 
regarding the plot to smuggle liquid explosive materials aboard the planes and assemble bombs. 

"TSA developed the Federal Flight Deck Officer Program as an additional layer of aviation security. 
See Pub. L. No. 107-296, § 1402(a), 1 16 Stat. 2135, 2300-05 (2002) (codified as amended at 49 
U.S.C. § 44912). Under the program, eligible flight crew members — a pilot, flight engineer, or 
navigator assigned to the flight — are authorized to use firearms to defend against an act of criminal 
violence or air piracy attempting to gain control of an aircraft. Federal Flight Deck Officers are trained 
by FAMS on the use of firearms, use of force, legal issues, defensive tactics, the psychology of 
survival, and program standard operating procedures. 

'Not applicable. 
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Table 4: OMB's 2003 and 2008 PART Review of FAMS— Detailed Assessment of Strategic Planning 

Answer 

Section 2 questions, explanation, and evidence 2003 2008 

Question 2.1 : Does the program have a limited number of specific long-term performance measures that focus Yes Yes 
on outcomes and meaningfully reflect the purpose of the program? 

Explanation and Evidence: The 2008 assessment noted that FAMS developed two long-term outcome 
measures: 

The number of successful terrorist and other criminal attacks initiated from commercial passenger aircraft 
cabins on flights in identified risk categories. 

The level of public confidence in air marshals' ability to promote aviation security. 



Question 2.2: Does the program have ambitious targets and time frames for its long-term measures? No Yes 

Explanation and Evidence: According to the 2003 assessment, FAMS's long-term measures were under 
development. The 2008 assessment recognized FAMS's two defined targets: 

The number of successful terrorist and other criminal attacks initiated from commercial passenger aircraft 
cabins on flights in identified risk categories. OMB noted that this target was ambitious because it 
quantified zero for all time frames. 

The level of public confidence in air marshals' ability to promote aviation security. OMB noted that FAMS 
quantified ambitious targets for a 5-year time frame. 



Question 2.3: Does the program have a limited number of specific annual performance measures that Yes Yes 

demonstrate progress toward achieving the program's long-term measures? 

Explanation and Evidence: The 2008 assessment stated that FAMS has two annual measures: 

The average annual rate of accuracy in air marshals' firearms requalification. Because a very high level of 
firearms proficiency is required for air marshals to successfully defeat terrorist and criminal attacks 
onboard commercial passenger aircraft, the associated rate of accuracy is one proxy outcome for the 
number of successful attacks. 

The percentage level in meeting FAMS's coverage targets for each individual category of identified risk. 3 
With U.S. air carriers providing over 29,000 flights daily, the ability of FAMS to identify high-risk flights for 
air marshal deployment is one proxy outcome for the number of successful attacks. 



Question 2.4: Does the program have baselines and ambitious targets and time frames for its annual No Yes 

measures? 

Explanation and Evidence: According to the 2003 assessment, FAMS's annual measures were under 
development. The 2008 assessment noted the following: 

FAMS has the highest firearm requirement for rate of accuracy (85 percent) of any federal law 
enforcement agency. No other federal law enforcement agency is known to afford its officers a smaller 
margin of error (15 percent) on its agencywide course of fire. Further, FAMS strives for an even higher 
standard— that is, to achieve an average annual practical pistol course score of 95 percent among all air 
marshals. 

FAMS established baselines and annual targets to measure the percentage of flights (in the various risk 
categories) covered by air marshals. The risk categories are based on vulnerability, threat, and 
consequence. The flight-coverage targets are highly ambitious (never striving for less than 100 percent) 
because FAMS, in accordance with its concept of operations, pursues flight coverage in the risk categories 
as a high priority. 
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Answer 


Section 2 questions, explanation, and evidence 


2003 


2008 


Question 2.5: Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, etc.) 
commit to and work toward the annual and/or long-term goals of the program? 


Yes 


Yes 


Explanation and Evidence: The 2008 assessment noted the following: 






• Air carriers and foreign air marshal programs are key partners upon whom FAMS relies to support 
performance goals. FAMS interacts daily with the airlines to ensure that both planned and emergent 
reservation requirements are met in order to ensure both ticketing and tactically optimal seating for air 
marshals on high-risk flights. 






Section 4017 of the National Intelligence Reform Act of 2004" encouraged President Bush to pursue 
aggressively international agreements with foreign governments to allow the maximum deployment of air 
marshals on international flights. As a consequence of numerous such bilateral agreements, U.S. air 
marshals cover U.S. flights into and out of multiple foreign countries. In some instances, the partner 
country has air marshals, whose deployment into the United States on foreign airlines is also provided for 
in the bilateral agreement. These bilateral partnerships reduce the threat of terrorism against aircraft of 
both nations. 






Question 2.6: Are independent and quality evaluations of sufficient scope and quality conducted on a regular 
basis or as needed to support program improvements and evaluate effectiveness and relevance to the 
problem, interest, or need? 


NA° 


Yes 


Explanation and Evidence: According to the 2003 assessment, FAMS had not been in existence long enough 
for its effectiveness to be assessed. OMB recommended that FAMS obtain an independent evaluation of its 
methods for risk analysis and resource allocation. The 2008 assessment noted that FAMS contracted with the 
Homeland Security Institute for an evaluation and that the institute's report, titled Federal Air Marshal Service 
Risk Assessment, was completed in July 2006. According to OMB, the institute's independent evaluation was 
comprehensive in its scope. 






Question 2.7: Are budget requests explicitly tied to accomplishment of the annual and long-term performance 
goals and are the resource needs presented in a complete and transparent manner in the program's budget? 


NA° 


No 



Explanation and Evidence: According to the 2003 assessment, FAMS had not been in existence long enough 
to present the resource needs in a complete and transparent manner in FAMS's budget. According to the 2008 
assessment, FAMS did not yet meet the requirement to put forth budget justifications that define explicitly the 
relationship between increments/decrements in program performance and increases/decreases in funding 
levels. OMB noted that FAMS is working with independent experts of the Homeland Security Institute to 
develop a means of transparently quantifying and communicating the performance implications (outputs and 
outcomes) that various resource levels (inputs) would drive. 



Page 47 



GAO-09-273 Federal Air Marshal Service 



Appendix II: OMB's Program Assessment 
Rating Tool Results in 2003 and 2008 for the 
Federal Air Marshal Service 





Answer 


Section 2 questions, explanation, and evidence 


2003 2008 



Question 2.8: Has the program taken meaningful steps to correct its strategic planning deficiencies? Yes Yes 



Explanation and Evidence: According to the 2003 assessment, FAMS received the rating of "results not 
demonstrated" because FAMS was unable to develop acceptable performance goals or collect data to 
determine whether it was performing. Specifically, OMB cited strategic planning deficiencies that included the 
absence of baselines, targets, and time frames associated with performance goals and performance 
measurements. Additionally, the 2003 assessment cited the absence of a second long-term outcome measure, 
proxy measures with respect to deterrence, and efficiency measures. 

According to the 2008 PART review, FAMS took steps to correct these strategic planning deficiencies by 
developing the following: 

A second long-term outcome measure — the level of public confidence in air marshals' ability to promote 
aviation security — which is reflective of FAMS's purpose. 

Proxy measures of deterrence, such as air marshals' average annual rate of accuracy in firearms 
requalification testing. 

Efficiency measures, such as (a) cost per flight per air marshal and (b) percentage of air marshals meeting 
the targeted number of flying days per year. 

Baselines, targets, and timelines associated with performance goals and performance measurements. 



Source: OMB's 2003 and 2008 PART reviews of FAMS. 

"In light of recent OMB guidance regarding the identification and constitution of proxy measures, 
FAMS believes that the coverage for categories of risk, while not a stand-alone outcome measure, is 
actually a "proxy" outcome for the number of successful attacks measure. That is, the ability to 
identify and deploy air marshals on higher risk flights reduces the likelihood of a catastrophic 
outcome. 

"The provision cited is part of the Intelligence Reform and Terrorism Prevention Act of 2004. 
c Not applicable. 
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Table 5: OMB's 2003 and 2008 PART Review of FAMS— Detailed Assessment of Program Management 

Answer 

Section 3 questions, explanation, and evidence 2003 2008 

Question 3.1 : Does the agency regularly collect timely and credible performance information, including Yes Yes 

information from key program partners, and use it to manage the program and improve performance? 

Explanation and Evidence: According to the 2008 assessment, FAMS collects performance information every 
roster period (the period of mission scheduling that corresponds to two pay periods) and applies the optimal 
volume of air marshals where the most risk lies. Also, OMB noted that, with respect to partners, FAMS actively 
requests from international air marshal programs their increased coverage on foreign-flagged flights to the 
United States. 



Question 3.2: Are federal managers and program partners (grantees, sub-grantees, contractors, cost-sharing Yes Yes 
partners, etc.) held accountable for cost, schedule, and performance results? 

Explanation and Evidence: According to the 2008 assessment: 

FAMS's key managers are subject to the departmentally standardized DHS senior executive service pay- 
for-performance system. More specifically, the FAMS Director, Deputy Director, Assistant Directors, senior 
executive service-level Deputy Assistant Directors, and senior executive service-level Special-Agents-in- 
Charge each enter into an annual executive performance agreement under that system. 

The performance agreements serve as vehicles for identifying (in writing) the managers who are 
responsible for achieving key program results and for clearly defining each manager's standards of 
accountability for achieving those results. FAMS's contract awards and renewals do consider past 
performance. 



Question 3.3: Are all funds (federal and partners') obligated in a timely manner and spent for the intended NA a Yes 

purpose? 

Explanation and Evidence: According to the 2003 assessment, FAMS had not been in existence long enough 
to assess obligation data on a timely basis. The 2008 assessment noted the following: 

According to sample fund status reports, the fiscal year 2007 financial plan, and the unobligated balance 
chart, FAMS obligated its funds in a timely manner and for intended purposes. 

FAMS's financial analysts reviewed and certified all procurement requests to ensure that funds were 
available and that planned actions comply with established appropriation law standards. 

As a result of these steps, FAMS achieved annual obligation rates of at least 99 percent. 



Question 3.4: Does the program have procedures (e.g., competitive sourcing/cost comparisons, information Yes Yes 
technology improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness 
in program execution? 

Explanation and Evidence: The 2008 assessment noted that FAMS gauges efficiency and cost 
effectiveness by measuring the following: 

The percentage of federal air marshals providing the targeted number of flying days per year. 
The cost per flight per air marshal. 
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Answer 


Section 3 questions, explanation, and evidence 


2003 


2008 


Question 3.5: Does the program collaborate and coordinate effectively with related programs? 


Yes 


Yes 


Explanation and Evidence: The 2008 assessment noted the following: 






Air marshals, TSA security officers (i.e., behavior detection officers), and airport police departments use 
FAMS's Tactical Information Sharing System to report suspicious activities. 






• Additionally, managing overseas air marshals' deployments requires strong partnerships with both U.S. 
government entities (e.g., Department of State, U.S. Embassy, and TSA Transportation Security Area 
Representative) and each foreign locality's host government (particularly foreign law enforcement 
authorities). Partnerships address the diplomatic, security, and logistical matters required to accomplish 
FAMS's international mission deployments. 






Question 3.6: Does the program use strong financial management practices? 


Yes 


No 


Explanation and Evidence: According to the 2003 assessment, FAMS appeared to manage financial resources 
properly. The 2008 assessment stated that FAMS and TSA use strong financial management practices; 
however, in the annual DHS audit (for which DHS has contracted with KPMG), the auditors reported two TSA- 
wide material control weaknesses" for fiscal year 2007; one of these material control weaknesses, regarding 
undelivered order balances, involved FAMS. 






Question 3.7: Has the program taken meaningful steps to address its management deficiencies? 


Yes 


Yes 



Explanation and Evidence: The 2008 assessment noted that FAMS participates in a system for evaluating and 
correcting program-level management deficiencies. Specifically: 

The program's field offices undergo, on a 3-year cycle, inspections by the TSA Office of Inspection, whose 
mission is to ensure the integrity and effectiveness of TSA employees and programs. 

The inspectors determine whether management deficiencies are present, and recommend courses of 
action to address any deficiencies identified. 



Source: OMB's 2003 and 2008 PART reviews of FAMS. 

"Not applicable. 

"A material weakness is a significant deficiency or combination of significant deficiencies that results 
in more than a remote likelihood that a material misstatement of the financial statements will not be 
prevented or detected. 
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Table 6: OMB's 2003 and 2008 PART Review of FAMS— 


-Detailed Assessment of Program Results/Accountability 


Section 4 questions, explanation, and evidence 


Answer 
2003 2008 



Question 4.1 : Has the program demonstrated adequate progress in achieving its long-term outcome No Small extent 

performance goals? 

Explanation and Evidence: According to the 2003 assessment, FAMS had not been in existence long 
enough to assess its long-term outcome performance goals. The 2008 assessment noted that: 



FAMS made substantial and measurable progress in achieving its long-term performance goals to 
(1) detect, deter, and defeat hostile acts targeting U.S. air carriers, airports, passengers, and crews 
and (2) improve confidence in the security of our nation's civil aviation system. 

FAMS met its targets for the number of successful terrorist and other criminal attacks initiated from 
commercial passenger aircraft cabins on flights in identified risk categories. The absence of these 
successful attacks is the intended result of the program's activities and is of direct, significant 
benefit to the public. 

With respect to its public confidence goal, the program has some measurable evidence of public 
confidence in air marshals. 



Question 4.2: Does the program (including program partners) achieve its annual performance goals? No Small extent 

Explanation and Evidence: According to the 2003 assessment, FAMS had not been in existence long 
enough to achieve its annual performance goals. The 2008 assessment noted the following regarding 
annual performance goals: 

The percentage level in meeting FAMS's coverage targets for each individual category of identified 
risk; FAMS exceeded its target in fiscal year 2006 and performed under its target in fiscal year 
2007. 

The average annual rate of accuracy in federal air marshals' firearms requalification scores is a 
new measure, baselined in fiscal year 2007. FAMS met the target in the first and second quarters 
of fiscal year 2008. 



Question 4.3: Does the program demonstrate improved efficiencies or cost effectiveness in achieving No Large extent 

program performance goals each year? 

Explanation and Evidence: According to the 2003 assessment, FAMS had not been in existence long 
enough to demonstrate improved efficiencies or cost effectiveness. The 2008 assessment noted that 
FAMS demonstrated cost efficiency by optimizing the allocation of federal air marshal teams to 
missions. 



Question 4.4: Does the performance of this program compare favorably to other programs, including No Yes 
government, private, etc., that have a similar purpose and goals? 

Explanation and Evidence: According to the 2003 assessment, OMB directed FAMS to have an 
independent evaluation of its program performance with respect to flight coverage risk categories, the 
distribution of covered flights, and target levels of coverage. According to the 2008 assessment, the 
Homeland Security Institute's independent evaluation of FAMS considered other programs whose main 
business line is deterrence and/or protection via risk management. According to the institute, no other 
organizations that face a similar challenge apply significantly better tools or methodologies. 
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Federal Air Marshal Service 





Answer 


Section 4 questions, explanation, and evidence 2003 


2008 


Question 4.5: Do independent and quality evaluations of this program indicate that FAMS is effective No 


Large extent 


and achieving results? 





Explanation and Evidence: As noted previously, following the 2003 assessment, OMB directed FAMS to 
have an independent evaluation of program performance. The 2008 assessment noted that the 
Homeland Security Institute conducted an independent evaluation of FAMS in 2006 and determined 
that FAMS ensures effective risk reduction by applying a reasonable and valid approach to analyzing 
risk and allocating resources. Additionally, the assessment noted that an interagency steering group — 
convened by the Homeland Security Institute and consisting of representatives from an array of law 
enforcement and counter-terrorism agencies— also considered FAMS's concept of operations to be 
reasonable. 



Source: OMB's 2003 and 2008 PART reviews of FAMS. 
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The Homeland Security Institute, a federally funded research and 
development center, was established to assist the Department of 
Homeland Security in addressing relevant issues that require scientific, 
technical, and analytical expertise. The institute — after conducting an 
evaluation of the Federal Air Marshal Service's approach for achieving the 
agency's core mission of providing an onboard security presence for 
flights operated by U.S. commercial passenger air carriers — issued its final 
report in July 2006. 

This appendix presents quoted excerpts that substantially replicate the 
executive summary in the Homeland Security Institute's July 2006 report. 



"The Federal Air Marshal Service (FAMS) challenge to reduce risk in the 
aviation domain is daunting. U.S. commercial passenger carriers make 
roughly 28,000 domestic and international flights each day. These flights 
canvas the globe and originate, terminate, or fly in proximity to thousands 
of critical facilities. The FAMS must evaluate which flights it will defend 
and to what extent. It cannot cover every flight. 

"In response to an Office of Management and Budget (OMB) direction, the 
FAMS asked HSI [Homeland Security Institute] for an independent 
evaluation of its methods for analyzing risk and allocating resources. In 
particular, it asked HSI to determine if its risk management processes and 
the application of its concept of operations (CONOPS) to scheduled 
commercial flights were valid." 



"We defined 'validation' as a test of whether or not the FAMS risk 
management processes and the outcome of those processes are 
reasonable and consistent externally with stated guidance and internally 
with its own CONOPS. Our analysis involved three tasks. First, we 
examined the conceptual basis for the FAMS approach to risk analysis. 
Second, we examined the FAMS scheduling process and analyzed the 
output of that process in the form of 'coverage' data, i.e., when and where 
air marshals were deployed on flights. Third, we developed and employed 
a basic quantitative model to study the implications of alternative 
strategies for assigning resources." 



Background and 
Tasking for an 
Independent 
Evaluation 



Evaluation Approach 
Used by the 
Homeland Security 
Institute 



Page 53 



GAO-09-273 Federal Air Marshal Service 



Appendix III: Excerpts from the Homeland 
Security Institute's July 2006 Report 



Findings Reported by 
the Homeland 
Security Institute 



"Based on our analysis, we find that the FAMS applies a valid approach to 
analyzing risk and allocating resources. In particular, its approach is 
reasonable given the scarcity of resources and the guidance it has 
received. It assesses risk as a function of threat, vulnerability, and 
consequence and employs a filtering process along with an allocation tool 
to optimize resource allocation. Moreover, the FAMS seeks to strengthen 
risk management processes by improving its scheduling tools and 
analytical techniques. We did not find any other organizations that face a 
similar challenge and apply significantly better methodologies or tools." 



Additional 
Considerations 
Reported by the 
Homeland Security 
Institute 



"During our analysis, we identified [the following] five issues that the 
FAMS should address itself or in conjunction with the Transportation 
Security Administration (TSA), the Department of Homeland Security 
(DHS), and the broader intelligence and security communities." 



Vulnerability "The FAMS definition of vulnerability ... is inconsistent with traditional 

risk-based definitions, which focus on the probability that an attack will 
succeed. It shifts the focus away from other potential vulnerabilities. We 
recommend that the FAMS reconsider its approach to vulnerability 
and engage the aviation security community on this issue." (The 
emphasis is in the original.) 



Filtering Process "The FAMS understanding of consequence and its subsequent 'filtering' 

process . . . bias its allocation decision. To focus limited resources, the 
FAMS filters flights according to . . . [various factors]. 

"Guidance in the form of legislation and departmental memoranda 
following 9/11 directed FAMS to focus on flights that present 'high security 
risks.' But, ultimately, that guidance was ambiguous and could be 
outdated. These fundamental assumptions concerning risk, on which 
it [FAMS] allocates resources, warrant interagency review by the 
broader intelligence and security community." (The emphasis is in the 
original.) 
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Allocation Process "The FAMS filtering process defines 'high risk' and directs its efforts 

toward flights fitting those characteristics. Its allocation process — a 
modified version of the SABRE software used by airlines to schedule flight 
crews — attempts to cover the maximum number of high risk flights within 
fixed resources. . . . The scheduling tool requires manual involvement to 
recognize and modify scheduling solutions, which may not be consistent 
with effective risk reduction. 

"Our analysis of one month of FAMS coverage data reveals . . . [some 
concerns.] To compensate for a lack of resources and deny 
predictability, the FAMS should integrate randomness into its 
allocations." (The emphasis is in the original.) 

"Contrary to the popular use of the term 'random,' allocating resources in 
such a way does not mean choosing them haphazardly or without a plan. 
The overall probability distribution for a group of comparable aircraft can 
be chosen based on risk analysis. For instance, the FAMS may choose to 
cover [a determined percentage of] frights in and out of [a particular 
geographic region]. But the tactical allocation decision concerning a 
specific flight must be random and converge around the overall category 
average over time. A terrorist group may be able to discern the overall 
category average through effective, long-term surveillance but will never 
know conclusively whether or not the flight it plans to hijack will be 
covered on a particular day." 



Performance Measures "The FAMS primary performance measure — average coverage rates — can 

mask weaknesses in coverage patterns. In particular, they can mask a 
situation in which certain flights within a category of comparable flights 
are heavily covered while others are rarely if ever covered. Accordingly, 
the FAMS should develop performance measures to track coverage 
consistency. One example involves tracking coverage deviation, defined 
as the average difference between the individual coverage rates of each 
flight in a comparable category and the overall category coverage rate." 
(The emphasis is in the original.) 



Decision Support Tools "During the course of our analysis, we noted that FAMS decision makers 

did not have a tool for evaluating the resource implications of different 
threat scenarios and alternative coverage schemes. The SABRE scheduler 
is not flexible enough to support quick-response analyses. The FAMS 
should build a simple decision-support tool, along the lines of the 
model we developed based on risk balancing, to facilitate a system- 
wide view of resource decisions." (The emphasis is in the original.) 
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"Such a tool would augment, not replace its scheduling tool, by allowing 
decision makers to look across the entire aviation system and investigate 
the resource implications of alternative allocation scenarios. In other 
words, how easily could the FAMS adapt to a different threat 
environment? Could it significantly increase the number of marshals 
aboard specific flights? Where might those resources come from? How 
would these changes affect coverage elsewhere?" 
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In 2003, the Office of Management and Budget (OMB) used its Program 
Assessment Rating Tool (PART) to assess the management and 
performance of the Federal Air Marshal Service (FAMS). 1 At that time, a 
key performance measure for assessing FAMS was based on coverage of 
targeted critical flights under various risk categories. This measure is still 
applicable currently; however, its designation has been changed from an 
output measure 2 to an outcome measure. 3 Moreover, this performance 
measure — the coverage of targeted critical flights — is now considered a 
proxy indicator 4 regarding air marshals' ability to defeat an attempted 
attack. Also, in further response to the findings of OMB's 2003 assessment, 
FAMS established two additional outcome measures, one of which serves 
as another proxy indicator of air marshals' ability to defeat an attempted 
attack: 

• The additional proxy outcome measure is the average annual rate of 
accuracy in air marshals' firearms requalification testing. 

• The additional, non-proxy outcome measure is based on a national survey 
of households to determine the level of public confidence in air marshals' 
ability to promote aviation security. 

These updated measures have been approved by the Department of 
Homeland Security — and also were approved in 2008 by OMB during its 
PART-based reassessment of FAMS. An overview of FAMS's updated 
performance measures is presented in table 7. 



1 PART consists of a standard series of questions intended to determine the strengths and 
weaknesses of federal programs. The PART questions cover four broad topics — (1) 
program purpose and design, (2) strategic planning, (3) program management, and (4) 
program results/accountability. See GAO, Program Evaluation: OMB's PART Reviews 
Increased Agencies' Attention to Improving Evidence of Program Results, GAO-06-67 
(Washington, D.C.: Oct. 28, 2005). 

2 OMB defines an "output" measure as one pertaining to the internal activities of a program, 
such as the products or services delivered. 

3 Regarding types of performance measures, OMB defines an "outcome" measure as 
pertaining to the events or conditions of direct importance to the public/beneficiary that 
are external to the program. 

4 According to OMB, programs that have difficulty defining a quantifiable outcome measure 
may adopt a "proxy" outcome measure or indicator. These proxy measures need not meet 
the external event or condition criterion for a standalone outcome measure. The proxy 
measure need not be a public good unto itself, but rather be linked to the public good that 
is captured by the parent outcome measure — which is, in this case, the number of 
successful attacks. 
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Table 7: Overview of the Federal Air Marshal Service's Key Performance Measures (as of Fiscal Year 2008) 



Performance measure 



Type of 
measure 9 



Term 



Description 



Number of successful 
terrorist and other criminal 
attacks initiated from 
commercial passenger 
aircraft cabins in identified 
risk categories 



Outcome Long-term FAMS expects that, on 100 percent of covered flights, air marshals will 

detect and defeat terrorist or criminal attackers to prevent hostile 
control of the aircraft. By virtue of meeting this measure's targets, air 
marshals deter onboard attacks. The annual target for this measure is 
zero successful attacks. Since September 1 1 , 2001 , to date, no 
attacks have occurred on board covered flights. 



Percentage level in 
meeting coverage targets 
for each individual 
category of identified risk 



Outcome Annual This measure reflects the performance levels of FAMS coverage of 

targeted critical flights based upon a risk-based management 
approach involving impact (geographical location), vulnerability 
(aircraft destructive potential), threats, and intelligence relative to the 
availability of resources. FAMS deploys air marshals to targeted critical 
flights under 10 individual risk categories that are specified in FAMS's 
concept of operations. According to OMB, this outcome measure is a 
proxy indicator regarding the number of successful attacks 
performance measure, given that deployment of air marshals on 
higher risk flights reduces the likelihood of a catastrophic attack. For 
this measure, FAMS has established its target to be 100 percent 
coverage of identified high-risk flights in each risk category. FAMS 
exceeded this goal in fiscal year 2006 (101.7 percent) and fell short of 
the target in fiscal year 2007 (96.2 percent) because of an increased 
emphasis on international flights resulting from the August 2006 
terrorist plot to attack flights from the United Kingdom to the United 
States. 



Average annual rate of 
accuracy in federal air 
marshals' firearms 
requalification 



Outcome Annual As a requirement for the job, each individual air marshal must score at 

least 85 percent on quarterly firearms requalification testing. However, 
as a stretch or ambitious goal, FAMS established a baseline precision 
rate of 95 percent in 2007, and FAMS is retaining this rate as an 
annual goal through 201 1 . According to OMB, the firearms precision 
requirement is reflective of FAMS's ultimate ability to defeat an 
attempted attack. Thus, as a second proxy outcome measure, FAMS 
calculates an average, annualized precision rate that encompasses 
the firearms requalification scores of all air marshals." 



Level of public confidence Outcome Long- Beginning in fiscal year 2008, the U.S. Department of Transportation's 

in federal air marshals' term/annual Bureau of Transportation Statistics included in its National Household 

ability to promote aviation Transportation Survey a question regarding respondents' confidence 

security in federal air marshals' ability to defend an aircraft and passengers 

from individuals with hostile intentions. 
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Performance measure 


Type of 
measure 9 


Term 


Description 


Percentage of air 
marshals meeting the 
targeted number of flying 
days per year 


Efficiency 


Annual 


In the spring of 2007, FAMS implemented a policy to target the 
number of flying days per air marshal to 18 days per 28-day roster 
period and 200 days per year. As a measure of efficiency for fiscal 
year 2007, FAMS established a target of 75 percent of all air marshals 
meeting the expected number of annual flying days; the target is to 
increase to 77.5 percent by fiscal year 201 1 . According to OMB, the 
actual rate achieved in fiscal year 2007 was 74 percent. 


Cost per flight per air 
marshal 


Efficiency 


Long- 
term/annual 


This measure reflects FAMS's total annual costs (full-time equivalent 
personnel costs, per diem costs, and program management costs) 



divided by the number of flights with an on-board air marshal presence 
during the year and further divided by the number of air marshals per 
flight. The result is the cost to the government per flight per air 
marshal. 



Source: Office of Management and Budget. 

"OMB defines an "efficiency" measure as one that captures a program's ability to carry out its 
activities and achieve results (an outcome or output) relative to resources (an input such as cost). 

"The firearms requalification proxy measure is reflective of the ability of air marshals to defeat an 
attack on board an aircraft, which is the parent outcome measure. 

c OMB did not publish the targeted and actual costs making up this efficiency measure because the 
data are considered sensitive security information. 
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In March 2006, the Director of the Federal Air Marshal Service (FAMS) 
communicated to employees his intention to establish working groups to 
examine a variety of issues ranging from mission, organizational, and 
operational topics to workforce satisfaction and quality-of-life concerns. 
Two months later, in May 2006, the director communicated to FAMS 
employees that 12 working groups had been established, with each chaired 
by a field office special agent-in-charge (SAC) and that subject matter 
experts from the field and headquarters were available to assist in an 
advisory role. 

Subsequently, the number of working groups increased to a total of 36. 
Table 8 categorizes the 36 working groups and briefly summarizes the 
purposes of each. 

Also, regarding the status of the 36 working groups as of October 1, 2008, 
FAMS officials reported the following (see notes to table 8): 

• 18 working groups (table note a): Each of these working groups had 
completed its work and given a final report to FAMS management. Each 
report had been reviewed by FAMS executives and then distributed to 
agency employees via a broadcast message from the FAMS Director. If 
applicable, the broadcast message also presented management's responses 
to any recommendations made by the respective report. 

• 9 working groups (table note b): Each of these working groups had 
completed its work and given a final report to FAMS management. The 
reports were undergoing review by FAMS executives. 

• 5 working groups (table note c): Each of these working groups had yet to 
complete its work and give a final report to FAMS management. 

• 4 working groups (note d): Each of these working groups is to remain 
ongoing. As such, final reports are not expected to be issued; rather, each 
group will present its findings when applicable and by appropriate means. 
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Table 8: Overview of Federal Air Marshal Service Working Groups and Their Purposes 



Issue categories Working groups and their purposes (first meeting date) 



Mission, 
organizational, 
and operational 
issues 



Corporate Analysis Working Group 3 (April 4, 2006) 

Review and evaluate the current processes and structure of FAMS. 

Develop a recommended organizational structure to address a broader range of responsibilities transferred to 
FAMS based on the recent Transportation Security Administration functional integration effort. 

Position the organization to achieve its strategic objectives over a 5-year period. 

Building a Law Enforcement Culture Working Group " (May 3, 2006) 

Assess FAMS's organizational culture. 

Determine if existing culture meets expectations of employees and stakeholders. 

Develop recommendations to promote and sustain a culture that represents the values, customs, and 
traditions that best reflect organizational expectations. 

Provide recommendations to best position FAMS for sustained growth and operations during the next 5-year 
period. 

Process Management Working Group 3 (May 1 6, 2006) 

Review and analyze the current process management components of FAMS to include strategic planning 
efforts, investment management, financial management, and administrative practices. 

Operational Management Issues Working Group " (May 1 7, 2006) 

Evaluate operational options in acknowledgment of the expanded role of FAMS within the Department of 
Homeland Security and the Transportation Security Administration, in the context of risk-based management 
and in coordination with federal, state, and local law enforcement counterparts. 

Project Management Working Group " (May 24, 2006) 

Examine the project management process and the tools and technology available to improve or enhance the 
process. 

Examine how goals, objectives, and strategies are identified and how milestones are established and 
monitored. 

Tactical Policy Working Group 3 (June 21 , 2006) 

Examine and review FAMS's tactical policies and procedures to include boarding procedures, dress code 
policies, transit within the airport environment, and hotel-choice policies for overnight stays. 

Develop enhancements or revisions that are needed to FAMS's current policies or operational procedures. 

Federal Flight Deck Officer Working Group " (August 1 , 2006) 

Review and analyze the existing focus committee recommendations for credentials/badges, checkpoint 
requirements, weapon issues (including transport, storage, and qualification), communication protocols, 
training, liaison issues with the airline associations and carriers, and develop associated implementation 
strategies. 

Firearms Working Group 3 (September 6, 2006) 

Conduct research and analysis to define the requirements for a future firearm acquisition to include the 
possibility of recommending an alternative firearm or a firearm to accommodate the range of federal air 
marshal physical sizes. 

Review the ammunition that is currently used for the duty firearm. 
Examine the off-duty firearm requirements and policy. 

Review the current firearm policies and procedures and determine if any revisions are required. 

Training Manual Working Group (October 24, 2006) 

Examine various training instruction documentation and, if appropriate, recurrent instruction practices at select 
field offices. 
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Issue categories Working groups and their purposes (first meeting date) 

Explosives Security Specialist Working Group " (October 31 , 2006) 

Discuss the explosives security specialists' current duties, responsibilities, and training requirements and any 
other associated issues within the Explosives Division and the field offices. 

Determine if any of the recently acquired programs within FAMS would benefit from involvement from the 
explosives specialists and, in turn, benefit the explosives specialists with enhanced experience. 

Develop a series of recommended proposals to further define the explosives security specialists' duties and 
responsibilities. 

Canine Program Working Group " (December 5, 2006) 

Review and discuss canine program operations, including the following elements: (1) canine team composition 
and operations for deployment at applicable airports; (2) cooperative agreement content; (3) roles of the 
Federal Security Directors, 8 the Assistant Federal Security Director for Law Enforcement, and SACs regarding 
canine team operations and coordination; and (4) additional issues as determined by working group members. 
SAC Advisory Council Working Group " (December 6, 2006) 

Examine and discuss a wide range of issues that impact FAMS's operations and provide recommended 
actions to the FAMS executive staff. [Note: Agency documentation explained that members of the SAC 
Advisory Council are uniquely positioned to provide extensive knowledge of operational issues that are 
critically important to executing the mission of FAMS.] 

Explosives Security Specialist Sub-Working Group " (August 28, 2008) 

Review recommendations previously made by the Explosives Security Specialist Working Group. 

International Mission Working Group " (February 6, 2008) 

Examine and discuss FAMS's international deployment procedures and concepts currently in use. 

Joint Terrorism Task Force Working Group " (March 6, 2008) 

Examine and discuss the various procedures and issues pertaining to FAMS personnel serving on Joint 
Terrorism Task Force assignments. 

Scheduling Practices and Operations Working Group (8/5/08) 

Review and examine various scheduling practices and protocols. 



Technology Technology Working Group " (May 24, 2006) 

issues • Examine FAMS's technological initiatives and requirements to ensure that the organization's needs are met 

now and in the future. 

Operational Communications Group " (November 9, 2007) 

Examine FAMS's operational and technical requirements for land mobile radio communications to support a 
number of initiatives, including Visible Intermodal Prevention and Response teams and other operations. 



Human capital Workforce Satisfaction, Recruitment, and Retention Working Group " (April 12, 2006) 

lssues • Examine FAMS's recruitment and retention practices and develop strategies and recommendations to 

enhance those efforts. 

Identify key issues that affect FAMS's employee workforce satisfaction and morale issues as the organization 
continues to mature over a 5-year period. 

Quality of Life, Scheduling, and Transfers Working Group " (April 25, 2006) 

Examine the quality-of-life and associated issues involving mission scheduling and manpower issues. 
Examine the current transfer/change-of-station program. 

Career Path and Promotional Opportunities Working Group " (May 2, 2006) 
Examine and assess the career path/promotional opportunity process for the federal air marshal and civilian 
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Issue categories Working groups and their purposes (first meeting date) 

population of FAMS. 

Identify recommendations to further develop/enhance career development and promotional opportunities 
within the organization. 

Examine managerial succession planning efforts and develop recommendations to best position FAMS for 
sustained growth and operations during the next 5-year period. 

Recruitment and Retention of Women & Family-Friendly Initiatives Working Group 8 (Aug. 9, 2006) 
Identify and analyze those issues that impact the recruitment and retention of women. 
Identify creative family-friendly initiatives that will positively impact quality of life and retention of employees. 

Mission Support Employee Working Group " (October 24, 2006) 

Examine various issues and policies that specifically pertain to civilians and range from the current structure of 
mission support employee positions to professional development training to enhance recruitment and 
retention. 

Hardship Transfer Policy Working Group " (October 25, 2006) 

Develop a hardship transfer policy that is (1 ) consistent, fair, and non-discriminatory and (2) tailored for the 
needs of a law enforcement organization and its officers. 

Awards and Recognition Working Group 3 (November 7, 2006) 

Review the current awards and recognition programs and policies of the Transportation Security 
Administration and FAMS. 

Develop recommended actions, as appropriate, to modify the current policies and practices. 

Diversity Working Group 3 (January 17, 2007) 

Discuss and examine the range of diversity-related concepts and issues that impact FAMS's personnel and 
operations. 

Focus on the continued development of a culture within FAMS that leverages the diversity of the workforce 
and identifies and utilizes the range of employee talent to carry out FAMS' mission/operations. 

Employee Outreach Working Group (February 2, 2007) 

Examine critical incident stress management models for law enforcement to explore their potential benefit for 
implementation within FAMS. 

Analyze traumatic incidents (specifically suicides) that have occurred to date within FAMS to identify 
commonalities and lessons learned across the cases, if any. 

Develop specific goals, objectives, and parameters associated with an organization-wide resource. 
Consult relevant scientific literature and professional expertise. 
Communications Communications Working Group 3 (June 13, 2006) 

and relationships . Review and evaluate the current communication and media relationships practices (from headquarters to the 
lssues field offices, within and among headquarters directorates, and within and among field offices). 

Gather input from the other working groups that have addressed communication issues and prepare 
recommendations to design communication and media practices and strategies for FAMS. 

Stakeholder Relationships Working Group " (June 14, 2006) 

Review and analyze the current stakeholder relationships with entities such as Transportation Security 
Administration, Department of Homeland Security, Office of Management and Budget, Congress, and various 
law enforcement entities. 

Determine how FAMS can best leverage those engagements to ensure a viable, sustainable organization. 

Assistant Federal Security Director for Law Enforcement Working Group 3 (June 27, 2006) 

Evaluate the current work activities of the position's workforce and then identify the core roles and 
responsibilities of that position. 

Codify the Assistant Federal Security Director for Law Enforcement's reporting procedures and the 
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Issue categories Working groups and their purposes (first meeting date) 

relationships between that position, the Federal Security Directors, and the field office SACs. 

Federal Air Marshal Issue Working Group 8 (October 3, 2006) 

Review current federal air marshal issue groups within FAMS and benchmark against similar types that are 
operational in other law enforcement organizations. 

Prepare a federal air marshal issue group charter that details objectives, membership, and standard operating 
practices and procedures to initiate and sustain a group within each field office. 

Policy Issues Working Group " (January 16, 2007) 

Identify and resolve conflicts between FAMS's policies (including legacy policies) and Transportation Security 
Administration management directives. [Note: This working group has transitioned to having periodic meetings 
between FAMS and TSA officials. No final report will be issued.] 

Integrated Conflict Management System Working Group " (August 14, 2007) 

Develop a proposal for implementing (including training) an integrated conflict management system within 
FAMS. 

Ensure that the proposal considers practical, operational, and organizational culture factors and also 
recognizes practices that are already in place. 

Propose recommendations concerning the roles and responsibilities associated with system coordinators' 
functions at the field offices and headquarters' divisions. 



Medical issues Medical Issues Working Group " (October 1 1 , 2006) 

Examine and review medical studies pertaining to the effects of flying in the aviation environment. 

Examine the number and types of claims for injuries and illnesses as documented in FAMS' Office of Workers' 
Compensation Program. 

Identify and recommend best practices from the military and aviation industry. 



Administrative Administrative Manual Working Group (November 7,2006) 

lssues • Develop an administrative manual to provide a resource for FAMS standard operating procedures and 

policies. 

Time and Attendance Forms Working Group 8 (April 1 9, 2007) 

Discuss and examine time and attendance sheets currently in use by all FAMS field offices. 

Make recommendations concerning which form(s) the group identified as a best practice to be replicated 
service wide — or, if no such form exits, make recommendations concerning a new form or modification to an 
existing one. 

Discuss and make recommendations concerning methods by which the time and attendance reporting process 
could become more streamlined and ensure consistency regarding reporting and pay. 



Source: Federal Air Marshal Service. 

According to FAMS officials, as of October 1, 2008, (1) the working group had completed its work 
and given a final report to FAMS management, (2) the report had been reviewed by FAMS executives 
and then distributed to agency employees via a broadcast message from the FAMS Director, and (3) 
if applicable, the broadcast message also presented management's responses to any 
recommendations made by the report. 

"According to FAMS officials, as of October 1, 2008, the working group had completed its work and 
given a final report to FAMS management, and the report was undergoing review by FAMS 
executives. 

■According to FAMS officials, as of October 1 , 2008, the working group had yet to complete its work 
and give a final report to FAMS management. 
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"According to FAMS officials, the working group is to remain ongoing, and a final report is not 
expected to be issued. Rather, the group will present its findings when applicable and by appropriate 
means. 

6 A Federal Security Director is the top-ranking TSA official responsible for security at each of the 
nation's commercial airports. 
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FAMS Workforce Satisfaction Survey 

On behalf of the Director and the Office of Law Enforcement [OLEJ/FAMS Office of 
Workforce Planning and Management, thank you for taking the time to complete this 
survey. The purpose of this survey is to capture information regarding workforce 
satisfaction at OLE/FAMS. The information obtained by this survey will be used by 
OLE/FAMS leadership to assess the current levels of workforce satisfaction for the 
purposes of planning, policy development, and program enhancement. In addition, the 
data obtained by this survey will be used to evaluate current OLE/FAMS workforce 
satisfaction initiatives and strategies. Please take the time to carefully complete this 
survey. Your input will remain confidential and is vital to making OLE/FAMS a premier 
law enforcement organization. Thank you for your help. 



For each item, choose the response that best reflects your experience at OLE/FAMS. 

/// / 



1 . 1 have trust in the abilities of OLE/FAMS senior leaders 
and executives (e.g., Director, Deputy Director, Assistant 
Directors, Deputy Assistant Directors) to lead the 
organization. 

2. OLE/FAMS executive leaders are receptive to 
organizational change. 



a □ □ □ □ 



□ □ □ □ □ 



3. I believe that the Director's visions and initiatives are 
consistently shared and supported by my field management. 
Supervisor/Management 



□ □ □ □ □ 



4. My first-line supervisor is competent in doing his/her 
job. 



5. My supervisor is receptive to change. 



6. My discussions with my supervisor about my 
performance are worthwhile. 



□ □ □ □ □ 



7. I have discussed a career development plan with my 
supervisor. 



□ □ □ □ □ 



9. 1 have adequate equipment, supplies, and materials to 
accomplish my duties. 


□ □ □ □ □ 


10. Generally, the equipment I use (e.g., firearm, computer, 
cell phone, PDA [Personal Digital Assistant], etc.) to 
perform my job works properly. 


□ □ □ □ □ 


1 1 . The equipment 1 use is sufficiently easy to operate. 


□ □ □ □ □ 


12. 1 am generally satisfied with the quality of OLE/FAMS 
physical facilities (e.g., workspaces, training facilities, 


□ □ □ □ □ 
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physical fitness areas, firearms ranges, etc.) 




13. 1 am generally satisfied with the availability of 
OLE/FAMS physical facilities (e.g., workspaces, training 
facilities, physical fitness areas, firearms ranges, etc.) 


I — I I — I I — I I — I I — I 

u u u u u 


14. I receive the training 1 need to do my job. 


□ □ □ □ □ 


15. 1 am satisfied with the frequency/amount of training I 
receive in my office. 


□ □ □ □ □ 


1 6. Generally, 1 am satisfied with the content and variety of 
job-related training I receive in my office. 


□ □ □ □ □ 


17. I am satisfied with the continuing education 
opportunities offered by my job. 


□ □ □ □ □ 


18. OLE/FAMS supports continuing education 
opportunities relevant to my job. 


□ □ □ □ □ 


19. My work schedule affords me the opportunity to pursue 
continuing education. 


□ □ □ □ □ 


20. I am encouraged by my supervisors and managers to 
seek training and educational opportunities. 


□ □ □ □ □ 




SA A N D SD 


2 1 . Overall, 1 am satisfied with the progress I have made 
toward my career goals. 


□ □ □ □ □ 


22. There are sufficient opportunities for career 
advancement at OLE/FAMS. 


□ □ □ □ □ 


23. In my present position, I have a clearly understood 
career path. 


□ □ □ □ □ 


24. Promotions to supervisory levels in OLE/FAMS are 
based on merit. 


□ □ □ □ □ 


25. In OLE/FAMS, the selection criteria for promotion are 
clear. 


□ □ □ □ □ 


Policies & Procedures 


SA A N D SD 


26. OLE/FAMS' written policies support (and do not 
hinder) mission accomplishment. 


□ □ □ □ □ 


27. The local policies and procedures of my office support 
mission accomplishment. 


□ □ □ □ □ 


28. I am able to stay updated and am informed about the 
latest policies and procedures. 


□ □ □ □ □ 
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Employee Involvement & Autonomy 


SA A N D SD 


29. I have effective channels to voice my opinion regarding 
work-related issues (e.g., working groups, listening 
sessions, e-mail suggestion box, etc.). 


□ □ □ □ □ 


30. I am empowered to use my professional discretion in 
daily execution of my duties. 


□ □ □ □ □ 


31. I am provided sufficient opportunities to participate in 
important decisions affecting my work. 


□ □ □ □ □ 


32. Outstanding performance is recognized in my office. 


□ □ □ □ □ 


33. 1 am satisfied with the promotion practices of 
OLE/FAMS. 


□ □ □ □ □ 


34. In my office, monetary rewards (i.e., cash awards, in- 
position increases, etc.) are tied to performance. 




















35. 1 am generally satisfied with my pay. 




















Communication 


SA A N D SD 


36. OLE/FAMS' policies and procedures are clearly 
communicated and easy to understand. 


□ □ □ □ □ 


37. 1 am satisfied with communication within my office 
(e.g., FO [Field Office], branch, division, directorate). 


□ □ □ □ □ 


38. There are mechanisms in place which allow me to 
freely express my comments, concerns, and suggestions 
without fear of retaliation. 


□ □ □ □ □ 


39. I am satisfied that the work-related concerns 1 address 
with management are addressed appropriately. 




















40. 1 have enough information to do my job well. 





















SA A N D SD 



41. I feel that my job-related stress is manageable. 


□ □ □ □ □ 


42. 1 am generally satisfied with OLE/FAMS programs 
related to employee safety, health, and wellness. 


□ □ □ □ □ 


43. 1 am satisfied that OLE/FAMS management is 
concerned for the health and safety of employees and is 
working continuously to offer improved services. 


□ □ □ □ □ 


44. I have been provided information and resources to take 
personal responsibility for my health and wellness as it 
relates to my job (e.g., proper diet, fitness, sufficient rest). 


□ □ □ □ □ 


45. I feel that medical information relevant to my job is 
communicated to me. 


□ □ □ □ □ 




SA A N D SD 


46. I am able to effectively balance my work with my 
personal/family life. 


□ □ □ □ □ 


47. My family is supportive of my career with OLE/FAMS. 


□ □ □ □ □ 


48. Current initiatives (e.g., Voluntary Lateral Transfer 
Program, new office openings, etc.) have a positive effect 
on quality of work life/family life. 


□ □ □ □ □ 
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49. OLE/FAMS leadership has implemented positive 
changes affecting scheduling. 


□ □ □ □ □ 


50. I am satisfied that OLE/FAMS is exploring initiatives 
to improve quality of life/family life. 


□ □ □ □ □ 


51. I have seen improvement in quality of work life and 
family life as a result of the recommendations from the 
Director's Working Groups. 


□ □ □ □ □ 


52. 1 have seen positive changes made in OLE/FAMS in the 
last year. 


□ □ □ □ □ 


Organizational Commitment 


SA A N D SD 


53. I am proud to work for OLE/FAMS. 




















54. I find my values are similar to OLE/FAMS values. 




















55. I feel a sense of loyalty to OLE/FAMS. 




















56. I am likely to stay at OLE/FAMS for the next 12 
months. 


□ □ □ □ □ 


Job Satisfaction 


SA A N D SD 


57. The work 1 do is important. 




















58. 1 find my work challenging and interesting. 




















59. Generally speaking, I am very satisfied with my job. 




















60. I like the kind of work I do (e.g., my current duties and 
assignment). 


□ □ □ □ □ 



Comments Section 

Please use this section to provide more specific information for any of the above 
questions. 



Reasons for Staying with OLE/FAMS 

Indicate the importance of each of the following factors in your reasons for staying with 
OLE/FAMS. 

Money (e.g., salary and benefits) 
Schedule (e.g., number of hours, flexibility) 

Duty location (e.g., FO, HQ [headquarters], SOCD [Systems Operation Control 
Division], etc.) 

OLE/FAMS mission (e.g., belief in importance of mission) 
Career Opportunities (e.g., promotion potential, career growth) 
Other: 
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tf& Department nrHiHncTnnd Security 
Wnahington H DC 

(jSftj Homeland 
^Hfp Security 

November 26, 2008 

Stephen M. Lord 

Acting Director. Homeland Security and Justice Issues 
U.S. Government Accountability Office 
441 G Street NW 
Washington, DC 20548 

Uear Mr. Lord: 

Thank you for the opportunity to review and provide comments on the Government 
Accountability Office ! s (G AO) draft report GAO-09 53SU titled, Federal Air Marshal 
Service Has Taken Actions to Fulfil! lis Core Mission and Address Workforce Issues, but 
Additional Actions Are Needed to Improve Workforce. Survey (44OS0I), The Department of 
Homeland Security (TJHS) appreciates OAO's effort in conducting this study and we believe 
that the findings and recommendation will facilitate continued progress in identifying and 
addressing issues lhal affect the ability of Federal Air Marshal Service (FAMS) personnel to 
perform the Agency's aviation security mi ssi on. 

In April 2007, GAO initiated its assessment of the FAMS. The inquiry lasted approximately 
1 7 months, concluding in October 200R. During this time. FAMS provided GAO with full 
access to FAMS* Executive Management, as well as access to the subject-maLlcr experts 
involved in the coordination of programs that affect the entire organisation. FAMS briefed 
GAO on numerous occasions regarding the 36 working groups and other initiatives that were 
implemented to address workforce issues. During diis period, GAO also conducted site visits 
and interviews at 1 1 of the KAMS* 2] field offices. During these visits, GAO had full access 
to the managers and was given access to interview air marshals of their choosing. This was 
the most in-depth inquiry conducted by an outside entity. 

DHS concurs with GAO'3 finding that the FAMS has successfully taken actions to fulfill core 
mission and aecepts the finding as validation of the efforts undertaken to improve FAMS 
morale and perrbrmaufe. 

Additionally, DHS/TSA is pleased to note GAO's eridorscrnenLs uf: 

* FAMS' Concept of Operation (consideration of risk-related factors deployed based on 
coverage goals and flexibility for responding to changing threats); 

• Management's efforts to address employee concerns and improve workforce 
satisfaction (Working Group Initiative, Field Office Focus Groups, Field Office Visits 
by the Director, Listening Sessions, Dinner/Breakfast with the Director, Director's and 



Page 70 



GAO-09-273 Federal Air Marshal Service 



Appendix VII: Comments from the 
Department of Homeland Security 



-2- 



Anonymous Suggestion In-Boxes, assignment of an agency Ombudsman, and 
establishment of the Voluntary Lateral Transfer Program); 
* Steps takeu to preserve anonymity of aii marshals (revision uf hotel and dress eode 
policies); and 

- Accomplishments in improving the quality of life for agency employees {Mission 
Exchange rrogram, 6 p.m. and 60-hour rule, limit on number of flights per days, more 
rest time following international missions, and more consistent start times). 

GAO made one recommendation that "the FAMS Director take appropriate actions to 
increase the usefulness of the workforce satisfaction surveys that the FAMS plans to conduct 
biennially. Such actions could include, for example, ensuring that the survey questions and 
the answer options are clearly structured and unambiguous and that additional efforts are 
considered for obtaining tlie highest possible response rates." 

We generally concur with the aforementioned recommendation. At present, FAMS is in the 
initial stages of formulating the next workforce satisfaction survey and plans to implement the 
suggestions. 

Unce again, DHS appreciates the opportunity to review and provide comments and we look- 
forward to working with you 011 future homeland security issues. 



Sincerely, 




.Terald E. Levine 
Director 

Departmental GAO/OIG Liaison Office 
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Contact Stephen M. Lord, Acting Director, Homeland Security and Justice Issues, 

(202) 512-4379 or lords@gao.gov 



Staff ^ n Edition t° t ne contacts named above, Danny Burton and John Hansen 

"' " (Assistant Directors) and Michael Harmond (Analyst-in-Charge) managed 

Acknowledgments this assignment. 

David Alexander, Chuck Bausell, Arturo Cornejo, Wendy Dye, Stuart 
Kaufman, and Courtney Reid made significant contributions to the work. 

Tom Lombardi provided legal support. 

Katherine Davis provided assistance in report preparation. 
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GAO's Mission 



Obtaining Copies of 
GAO Reports and 
Testimony 



The Government Accountability Office, the audit, evaluation, and 
investigative arm of Congress, exists to support Congress in meeting its 
constitutional responsibilities and to help improve the performance and 
accountability of the federal government for the American people. GAO 
examines the use of public funds; evaluates federal programs and policies; 
and provides analyses, recommendations, and other assistance to help 
Congress make informed oversight, policy, and funding decisions. GAO's 
commitment to good government is reflected in its core values of 
accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost 
is through GAO's Web site (www.gao.gov). Each weekday afternoon, GAO 
posts on its Web site newly released reports, testimony, and 
correspondence. To have GAO e-mail you a list of newly posted products, 
go to www.gao.gov and select "E-mail Updates." 



Order by Phone 



To Report Fraud, 
Waste, and Abuse in 
Federal Programs 

Congressional 
Relations 



The price of each GAO publication reflects GAO's actual cost of 
production and distribution and depends on the number of pages in the 
publication and whether the publication is printed in color or black and 
white. Pricing and ordering information is posted on GAO's Web site, 
http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 
MasterCard, Visa, check, or money order. Call for additional information. 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm 
E-mail: fraudnet@gao.gov 

Automated answering system: (800) 424-5454 or (202) 512-7470 

Ralph Dawn, Managing Director, dawnr@gao.gov, (202) 512-4400 
U.S. Government Accountability Office, 441 G Street NW, Room 7125 
Washington, DC 20548 



Public Affairs 



Chuck Young, Managing Director, youngcl@gao.gov, (202) 512-4800 
U.S. Government Accountability Office, 441 G Street NW, Room 7149 
Washington, DC 20548 



